Scott Scheff
Founder, Principal Human Factors Practitioner, HF Designworks
Image credit war.gov: National Guardsmen sit inside a ground control station for an RQ-7B Shadow unmanned aerial system
Autonomous systems are often touted as ways to reduce human effort and many of them do reduce the workload addressed by their requirements. Yet autonomy can shift work into activities that requirements may not capture, especially monitoring of automated systems that can erode vigilance over time. Routine work may be automated until the system encounters an exception outside its design envelope. Then the operator must intervene, and a crew ratio sized for nominal operations may not provide enough capacity. The central issue is exception handling: A ten-year cross-service review found that human factors were involved in 60% of military unmanned aircraft system (UAS) mishaps; latent failures at organizational, supervisory, or precondition levels contributed to more than half, well before the operator sat down at the control station.
The lessons below are not new. They come from fielded UAS, from unmanned surface vessels, from the automotive industry’s experience with supervised driving, and from supervisory control research. What is new is that the systems now being designed, from intelligent wingmen to unmanned surface and undersea fleets, are relying on one-to-many control at a scale earlier programs never attempted and could not have supported with the technology then available. As autonomy expands, programs need to understand both nominal workload and workload during off-nominal events.
Why workload is a program risk, not "just" a human factors detail
Operator workload is not just a topic for crew station design review. Operator workload is also a cost and schedule driver that surfaces in three critical places within a program. The first is manpower: the number of operators per vehicle, or vehicles per operator, is one assumption that strongly influences a system’s life-cycle cost, and it is usually set by an early concept of operations rather than by measurement. As an example, the Air Force’s remotely piloted aircraft fleet had fewer personnel than authorized from fiscal years 2016 through 2019, a shortfall GAO linked to the service’s ability to balance combat and noncombat duties and to concerns about burnout and retention.
The second is safety. A ten-year cross-service review of 221 military UAS mishaps found that 60% of mishaps involved human factors and the patterns of latent failures and unsafe acts differed across Air Force, Army, and Navy mishaps. Two recent UAS losses illustrate what that looks like in practice. In 2020 an MQ-9A operator pulled the fuel condition lever instead of the flap lever 7 seconds after takeoff, triggering the fuel shutoff valve, stopping the engine, and resulting in the aircraft crashing. The investigation board found that the two levers were roughly an inch apart, lacked labels, and were the same color. In February 2024 another MQ-9A was lost when the crew disengaged automatic takeoff at low altitude, a habit the unit had developed to satisfy airspace rules at their home station. Operators disengaged automation 21 seconds after liftoff while the manual throttle remained at 0%. The crew noticed the loss of airspeed eight seconds later, but the pilot did not advance the throttle until 22 seconds after disengagement. The throttle command came one second before impact, too late for the engine to respond.
The third is requirements stability. DoD Instruction 5000.95 directs capability developers and program managers to plan and implement HSI from initial user requirements through system disposal. It also requires HSI-related and human performance requirements to include feasible metrics and calls for trade-off analyses across HSI domains so human performance data informs total system performance. Workload and its effects on total system performance need to be considered early. Programs that wait to consider workload (such as waiting until operational test) often find that if changes are required the program is at a more expensive stage of development.
Image credit United States Air Force: A U.S. Air Force Captain conducts a training flight with a MQ-9 at Hancock Field Air National Guard Base
Seven lessons from the field
LESSON 1
Autonomy doesn’t remove workload but relocates it
Lisanne Bainbridge described the core irony in 1983: as automation handles more routine control, the operator is left to monitor the system and intervene in abnormal conditions, even though people are poorly suited to sustained monitoring. Over forty years later, this still rings true. Programs that budget only for the work that disappears, and not for the new work that appears, can create crew-ratio problems.
lesson 2
Supervising and controlling are different jobs. Size the crew to the more difficult one
The number of vehicles one operator can handle depends on what “handle” means. In a multi-phase study with experienced operators, one operator could supervise up to 15 unmanned aircraft for health monitoring with moderate automation. In a more demanding mission-and-payload scenario, the limit was three systems, and only three of ten operators completed that scenario with any degree of success.
Supervisory control research formalizes this as "fan-out", which estimates capacity from the time each vehicle can be neglected relative to the interaction time it demands and then accounts for delays when several vehicles need attention at once. The practical lesson for systems engineers is that a one-to-N control requirement is meaningless without a stated task mix. Current collaborative combat aircraft concepts include scenarios in which a pilot directs four to ten aircraft. Such ratios may work while systems remain nominal, but when a supervisor must intervene, however, workload and situation-awareness demands rise sharply.
Human Factors of Off-Nominal Multi-UAS Operations: When Automation Needs Help
Read the HF Designworks white paper discussing human factors considerations for operator intervention of autonomous assets
Lesson 3
The dangerous moments are the transitions
Both MQ-9A mishaps cited earlier unfolded within the first minute after liftoff, during transitions between automated and manual control. They illustrate why handoffs deserve separate analysis. Endsley and Kiris showed in 1995 that operators who have been out of the loop take longer to detect a failure and regain manual control and this effect grows with the level of automation they were supervising.
Every autonomous system shares similar transitions: takeoff and recovery, mode changes, lost link, handover between shifts or between control stations, and the moment the autonomy hands a problem back to the human because it has run out of its design envelope. Programs should treat every transition as a named use case with its own workload measurement. Tests limited to transitions during nominal flight will reveal very little about exception workload.
Lesson 4
Optimize workload because both overload and underload create risk
Just as high workload can contribute to accidents, so too can low workload. Investigating a fatal 2019 collision in which a supervised-driving system drove into a crossing truck, the NTSB attributed the crash in part to the driver’s “inattention due to overreliance on automation.” The report also stated that drivers are poor at monitoring automation and do not perform well on tasks requiring passive vigilance, and noted that 7.7 seconds without hands on the wheel was too short to trigger any warning. When designing for automation, design for engagement: give the operator meaningful decisions at a cadence that keeps them in the loop, make the automation’s intent and confidence visible, and instrument the crew station so that disengagement is detected before it matters.
Lesson 5
Small interface decisions can be a program risk
The 2020 MQ-9A investigation board found that two levers within an inch of each other, without labels or color differentiation, substantially contributed to the loss of an aircraft. The board also wrote that the levers “could easily be mistaken by an inexperienced, fatigued, or confused crewmember.” In this mishap, fixation, lever proximity, and the lack of distinguishing cues combined to produce repeated control-identification errors. The same logic applies to the software interfaces that now dominate autonomy control stations. Mode annunciation, alert prioritization, the visibility of what the autonomy is about to do next, and the number of screen actions between noticing a problem and acting on it can affect workload and appear in mishap investigations. These are design decisions that are easier to address in a preliminary design than to retrofit them once fielded.
lesson 6
Crews will adapt to your design in ways you did not plan: be vigilant of workarounds
The 2024 UAS accident mentioned above occurred when the crew disengaged automatic takeoff at low altitude because that was the norm at their unit, a workaround that had developed because of a local airspace constraint. Workarounds are how operators reconcile a design that may have tested well in isolation but does not fit real operations. Workarounds are often invisible to the program until one of them fails. To understand workarounds, look for the adaptations operators make. Observe operators using the fielded or prototype system and compare their work with documented procedures. Ask what they do that the procedures do not say. Treat each workaround as a requirement or training finding.
lesson 7
Make workload a requirement and give it a metric
Workload competes with every other attribute for design margin, and it can be overlooked if it is not stated as a requirement. DoDI 5000.95 directs capability developers and program managers to plan and implement HSI from initial requirements through disposal, using feasible metrics and trade-offs across the human systems integration (HSI) domains. For workload, that can mean stating the crew ratio with a defined task mix and mission phase, defining maximum acceptable workload in measurable terms, and verifying the requirement through modeling, human-in-the-loop simulation, and developmental test. A verifiable workload requirement can be managed and enforced.
What program managers and system engineers can do now
Programs can improve workload decisions early without a large investment. A small set of questions asked early can expose assumptions before they harden into design constraints for the system.
The common thread is measurement. Workload is an attribute of an autonomous system that can be modeled before there is hardware, measured in simulation before there is a prototype, and verified in test with the same instrument used to write the requirement.
HF Designworks is a human factors and human systems integration firm that works with defense primes and government programs on the operator side of autonomy and complex systems: requirements development and tailoring, workload and crew-ratio assessment, control station and interface design, human-in-the-loop evaluation, and HSI planning under DoDI 5000.95.
HF Designworks maintains a small-business status within the DoW and has worked within the defense industry for over 20 years serving as both a prime and a sub on a variety of ground, sea, and aviation based programs including those for the Army, Air Force, Navy, DARPA and NASA. For more information or to schedule a consultation please contact us.
Sources
U.S. Government Accountability Office, Unmanned Aerial Systems: Air Force Should Take Additional Steps to Improve Aircrew Staffing and Support, GAO-20-320, June 2020.
Tvaryanas, A. P., Thompson, W. T., and Constable, S. H., U.S. Military Unmanned Aerial Vehicle Mishaps: Assessment of the Role of Human Factors Using HFACS, 311th Human Systems Wing, Brooks City-Base, TX, 2005 (DTIC ADA435063).
U.S. Air Force Accident Investigation Board report, MQ-9A 15-4295, 108th Attack Squadron, Syracuse, NY, 25 June 2020 (released April 2021).
U.S. Air Force Accident Investigation Board report, MQ-9A T/N 13-4231, U.S. Africa Command area of responsibility, 11 February 2024, released 27 September 2024.
Department of Defense, DoD Instruction 5000.95, Human Systems Integration in Defense Acquisition, 1 April 2022.
Bainbridge, L., “Ironies of Automation,” Automatica, vol. 19, no. 6, 1983.
Porat, T., Oron-Gilad, T., Rottem-Hovev, M., and Silbiger, J., “Supervising and Controlling Unmanned Systems: A Multi-Phase Study with Subject Matter Experts,” Frontiers in Psychology, 2016.
Cummings, M. L., and Mitchell, P. J., “Predicting Controller Capacity in Supervisory Control of Multiple UAVs,” IEEE Transactions on Systems, Man, and Cybernetics—Part A, vol. 38, no. 2, 2008.
Jensen, B., et al., Cockpit or Command Center? C2 Options for Collaborative Combat Aircraft, Center for Strategic and International Studies, October 2024.
Endsley, M. R., and Kiris, E. O., “The Out-of-the-Loop Performance Problem and Level of Control in Automation,” Human Factors, vol. 37, no. 2, 1995.
National Transportation Safety Board, Collision Between Car Operating with Partial Driving Automation and Truck-Tractor Semitrailer, Delray Beach, Florida, March 1, 2019, Highway Accident Brief HAB-20/01, 2020.
Contact
HF Designworks, Inc.
PO Box 19911
Boulder, CO 80308
(720) 362-7066