// CASE STUDY
HF Designworks developed FortiFly to evolve human workload assessment beyond a single subjective score, providing a real-time view and assessment of how operator demand changes throughout complex tasks, training, and human-autonomy interactions.
The Challenge: Workload Is Not Static
For decades, human factors researchers have studied workload to understand whether operators can safely and effectively perform increasingly complex tasks. But many traditional approaches have an inherent limitation: they tell you how demanding an experience was, not necessarily when that demand occurred.
For example, an operator may complete a 30-minute mission and then post-mission subjectively report elevated workload. That is useful information, but important questions are left unanswered. Was workload consistently high? Did workload spike during a particular task? Was the operator responding to an alert, managing multiple assets, interpreting new information, or interacting with automation? What periods of high workload were missed or forgotten during the after action review. For complex systems, all those moments matter.
In the case of unmanned aircraft systems (UAS), as autonomous systems become more advanced, operators must fuse and process an increasingly abundant amount of information from multiple sources while simultaneously supervising multiple assets and intervening where autonomy cannot resolve an issue. HF Designworks saw a need to better connect workload measurement with what was actually happening; second-by-second.
From a Single Workload Score to a Workload Timeline
Developed through a NAVAIR Small Business Innovation Research (SBIR) effort, FortiFly expands workload assessment from a post-task snapshot into a continuous view of operator demand. This is achieved by combining multiple physiological, behavioral, and performance-based inputs to estimate workload throughout a training session or mission. These inputs can include measures such as eye gaze, pupillometry, heart rate and heart-rate variability, speech, fine-motor activity, task performance, and operator interactions.
Rather than relying on any single signal as a proxy for workload or requiring personnel to self-report, FortiFly brings multiple sources together within a common workload framework to create a more complete picture of operator demand across time.
The result is not simply a determination that workload was “high” or “low.” With FortiFly, researchers, instructors, operators, system developers, and even the system itself can see how workload changes over time and then begin connecting those changes to specific events within the task or mission.
Putting Workload In Context
Measurement becomes enormously more useful when it can be tied back to what the operator was doing. FortiFly is designed to integrate workload data with the systems and interfaces being evaluated. FortiFly’s Automatic Widget Identification Component (AWIC), for example, can identify and track regions of interest within third-party interfaces, recording not only where the operator is looking, but what they are looking at. AWIC as part of FortiFly is an invaluable tool in helping researchers understand where operators are focusing and interacting as workload changes.
FortiFly.Rewind then adds yet another layer of context by synchronizing workload measurements with recorded sessions. Researchers and instructors can return to areas of interest and examine the task, interface interaction, or operational event occurring at a single moment in time.
With FortiFly, a workload peak is no longer just a point on a graph, it becomes a moment that can be further investigated and learned from. This distinction allows human factors teams to ask much more actionable questions including:
What was happening when workload increased?
Which tasks or interactions appear to contribute most heavily to operator demand?
How does workload change across operators, interfaces, mission conditions, or levels of automation?
Where might training, interface changes, or automation better support the operator?
FortiFly.Rewind then adds yet another layer of context by synchronizing workload measurements with recorded sessions. Researchers and instructors can return to areas of interest and examine the task, interface interaction, or operational event occurring at a single moment in time.
With FortiFly, a workload peak is no longer just a point on a graph, it becomes a moment that can be further investigated and learned from. This distinction allows human factors teams to ask much more actionable questions including:
What was happening when workload increased?
Which tasks or interactions appear to contribute most heavily to operator demand?
How does workload change across operators, interfaces, mission conditions, or levels of automation?
Where might training, interface changes, or automation better support the operator?
Building Confidence in Real-Time Measurement
Real-time workload data is valuable only if teams can have confidence that it meaningfully represents operator demand. As part of FortiFly's development, HF Designworks has evaluated its real-time workload algorithms against established workload assessment methods, including the NASA Task Load Index (NASA-TLX). This allows established subjective measures and new real-time data to inform one another rather than treating them as competing approaches.
FortiFly's underlying workload framework also separates demand across multiple dimensions, providing teams with more information than an overall workload value alone. Together, these methods enable a richer analysis of human performance: not only how much workload an operator experiences, but how that workload develops throughout an operation and where its demands originate.
Designed for the Future of Human-AI Teaming
FortiFly was initially developed to support future aviation concepts involving automation, artificial intelligence, multi-domain data fusion, and manned-unmanned teaming. These environments create a different kind of human factors challenge. For example, automation may reduce the need for continuous manual control while simultaneously increasing monitoring, decision-making, communication, and supervisory demands.
Understanding that balance is critical. Real-time workload assessment can help teams evaluate where automation is successfully reducing operator demand, where it may inadvertently be creating new demands, and where additional support may be needed.
The same capability can also be applied beyond the original aviation use case. FortiFly can support simulation-based training; comparative interface evaluation; human-in-the-loop research; multi-UAS testing; and other environments where understanding the relationship between people, technology, and performance is essential.
Turning Measurement into Better Decisions
The goal of workload assessment is not simply to generate another metric, it is to produce information that helps researchers make better decisions about systems and the people who operate them. By connecting workload to tasks, interactions, system events, and operator performance, FortiFly gives researchers a way to investigate the moments that matter most.
FortiFly is not just a research tool however. For training organizations, FortiFly enables identifying where operators need additional practice or support. For designers and engineers, findings from FortiFly can reveal interactions that deserve further investigation or redesign. For autonomy teams, FortiFly findings can help identify where automation should assist the operator and where human involvement remains essential.
And for increasingly complex human-machine systems, FortiFly provides something traditional workload snapshots cannot; a more comprehensive view of human performance as it unfolds.
Contact
HF Designworks, Inc.
PO Box 19911
Boulder, CO 80308
(720) 362-7066