Beyond Compliance: The Real Value of Human Systems Integration

History is abundant with programs, systems, and products that pass design reviews and perform well in the lab, then fail miserably upon use by actual operators in real conditions. When design fails to consider its end user(s) training takes longer than planned due to usability issues, operators create workarounds because of unanticipated design flaws. Interfaces that made sense to the engineering teams are difficult for operators to use in real operating environments.

In many cases, the individual engineering decisions made across different teams were defensible and made sense in isolation. The problem emerges in the way those decisions interact together as part of an integrated system Hardware, software, staffing, training, maintenance, and human performance may each be addressed, yet the total system still falls short in use.

Human Systems Integration (HSI) bridges the gap between design expectations and real world performance with end users.. DoD Instruction 5000.95 calls out HSI as an explicit expectation across defense acquisition. An intentional and empoweredHSI program should beginfrom initial user requirements and continue throughout the program lifecycle, even through system disassembly and disposal.

At HF Designworks, we have spent more than 20 years integrating human performance into complex systems across Naval aviation, Army operations, Air Force advanced teaming, unmanned aircraft, and safety-critical commercial products. Here's how we think when it comes to  doing HSI right, and where we can help.

The Scope of HSI

HSI is a disciplined systems engineering practice that optimizes total system performance by addressing seven domains (though these can vary slightly across defense and acquisition):

1. Human factors engineering

2. Manpower

3. Personnel

4. Training

5. Safety and occupational health

6. Force protection and survivability

7. Habitability

HSI considers the human as part of the overall system and brings hardware, software, and humans together to minimize lifecycle cost by addressing how decisions in one domain can create ripple effects in several others.

Consider a theoretical design change that requires an operator to perform a new monitoring task. This new task may increase workload or change how operators allocate their time and attention. The resulting workload may affect crewing assumptions and manpower requirements, which can impact training demands and required qualifications. 

Programs that address each domain in isolation often discover the interactions and ripple effects between decisions too late, when they're most expensive to fix and have the largest impact on the lifecycle cost of the system. HSI provides a way to identify the tradeoffs, document their implications, and give program leaders a more complete picture for making decisions. 

HSI Earns Its Keep Across the Lifecycle

Clients often ask when an HSI effort should begin. The answer is almost always as early as possible. In practice, HSI teams can contribute long before there is an interface, prototype, or fieldable system to test.

Early HSI activities can (and often should) include defining measurable human-performance requirements, analyzing operator and maintainer tasks and conducting cognitive task analyses to understand how end users actually work, identification of potential human errors, examining crewing and personnel assumptions, and comparing alternative allocations of work between people and automation. These analyses help a program understand what the mission will demand and whether the proposed system concept supports those demands. Getting a workload or manning assumption wrong at Milestone A can be a memo; discovering it at operational test is a program breach.

HSI operates inside the engineering rhythm

As the system takes shape, HSI becomes a continuous engineering collaborator and valuable partner in the design tradespace. Hazard identification and mitigation, human error analysis, interface design, and iterative evaluation with representative users can all be performed as the design matures. At HF Designworks we pair human factors expertise with user experience (UX) and product design experts so findings don't just land in a report, they integrate into  the design.

Standards are an important part of the HSI toolkit. For example, MIL-STD-1472H provides human-engineering design criteria, while MIL-STD-882E provides a playbook for system safety practices. The applicability of standards such as these depends on how they are invoked for a particular program or contract. Other program-specific standards from organizations including ASTM, FAA, FDA, OSHA, and SAE can also guide HSI work inside a program and inform requirements, design, and performance expectations. HF Designworks personnel stay current on these ever evolving standards in large part through leading several subgroups within these international organizations; including ones that develop human factors engineering standards for ground control station design. Note that while a standards review can identify relevant criteria,  it cannot replace the analysis needed to understand users, tasks, operating conditions, or tradeoffs.

The same principle applies to usability studies, heuristic evaluations, benchmark tests, style guides, and other familiar human-factors or UX activities. Each can be valuable when it can be driven by or inform a requirement, risk, or design question. Used as isolated deliverables however, while they may document issues, they won’t necessarily provide the engineering team with a clear path to address them. That is where companies such as HF Designworks can come in to support, leveraging a holistic plan to not just provide findings, but actionable recommendations as well. 

Along those lines, as HSI findings are generated, they need to enter the program's established decision channels. Depending on the program, that may include requirements repositories, systems engineering reviews, risk management processes, working groups, and test planning. The objective is to give human performance evidence the same traceability and visibility as other engineering evidence.

Build evidence before operational evaluation

Workload and human performance modeling are other tools in an HSI practitioners bag that can be especially useful when a program needs to evaluate a concept that is difficult or expensive to test physically. Simulating operator tasks and workload across mission scenarios allows a program to compare crewing concepts and automation strategies before committing to a final implementation. Detailed workload analysis provides program leadership with quantitative evidence for difficult decisions.

For example, HF Designworks has publicly described combining functional analysis, cognitive task analysis, and discrete-event modeling to examine crew and automation configurations for Army Future Vertical Lift concepts. The purpose of this kind of analysis is to make assumptions explicit and compare their effects under representative mission conditions. Modeling does not eliminate the need for testing, but it can help programs decide what to build and what later evaluations need to verify.

Human-in-the-loop studies and system evaluations can provide evidence from representative users performing realistic tasks. Measures might include task performance, error patterns, workload, situation awareness, or other outcomes tied to the program's risks and requirements. The appropriate measures depend on what the system must enable people to do; they should not become a standard checklist applied without an understanding of the operating concept.

These HSI processes and artifacts create a traceable thread from early human performance requirements to the analysis, design decision, and test evidence used to evaluate them. These processes also give a program more than a late usability finding. The evidence can show whether the underlying assumptions about operators, maintainers, automation, staffing, and training remain credible as the design evolves.

Continue through fielding and modernization

HSI still remains relevant after initial development. Fielded systems change through software updates, new missions, revised procedures, advances in automation, and changes in the user population or operating environment. Each change can reopen human performance questions that appeared resolved in the original design.

A revised interface may alter task timing. New automation can reduce one source of workload while creating additional monitoring or coordination demands. A change in maintenance procedures may introduce new training or personnel requirements. Evaluating these consequences during sustainment can identify usability regressions and training effects before they become widespread operational problems.

Acquisition pathway, operational risk, the roles assigned to operators and automation, and the maturity of available evidence should guide what HSI work is warranted. A mature subsystem with limited human interaction may only need a focused review. A new operational concept with significant crewing, workload, or safety implications may require sustained analysis and evaluation across several domains.

What a useful HSI effort leaves behind

Programs do not need HSI activity for its own sake. They need evidence that improves decisions and includes the end user in those decisions. A productive HSI effort should leave a program with clearer human performance requirements, traceable tradeoffs among HSI domains, documented risks and mitigations, design decisions informed by operational evidence, and an evaluation approach that traces back to the intended mission and users.

This is also a practical way to assess an existing HSI program. If findings remain separate from requirements, architecture, risk management, and test planning, the HSI effort may be recording human performance concerns without integrating them into the overall system. If the evidence is changing assumptions, clarifying tradeoffs, and shaping what the program builds and verifies, HSI is being properly implemented.

Whether you're standing up HSI on a new program, need targeted support for an upcoming milestone, or want an independent human performance assessment of a system in development, HF Designworks can help examine the program's specific decisions, risks, and evidence gaps. We support programs at every scale and budget, performing evaluation and analysis leveraging decades of experience performing everything from focused usability studies, evaluating systems based on UX related standards and policy, to embedded multi-year HSI efforts on products and systems that are fielded today.

Contact us and let’s talk about how we can assist with your HSI program.

Image credit https://www.af.mil/News/Photos/igphoto/2003967275/