Dynamic Assessment of Crew Workload for Improved Human/Autonomy Teaming in Naval Aviation Operations

As autonomy becomes more sophisticated, the role of humans is shifting to operator, supervisor, and in future concepts, teammate. As more autonomy is integrated into Navy operations, it is even more important to understand the human/autonomy (H/A) team, and the effect that actions taken by autonomous systems have on the workload of the human teammate and the performance of the team. Under the proposed research program, Barron Associates, Inc. (BAI) has teamed with University of North Dakota (UND) to develop and validate HARMONY (Human Autonomy Response Monitoring fOr NavY operations), a system that measures and encapsulates the multidimensional nature of human workload, so that tasks can be properly allocated between both members of the H/A team.HARMONY will initially be targeted at crew conducting naval aviation operations including Naval Aviators, Naval Flight Officers, Naval Aircrewmen, and Naval Aerial Vehicle Operators. The team recognizes that with advancements in Artificial Intelligence and Machine Learning (AI/ML), the tasks that can be accomplished with autonomy are changing more rapidly than ever before. As such, HARMONY is designed to be agnostic to the autonomy as much as possible. Instead, HARMONY will output a valid and consistent metric that captures the current state of the human teammate compared to benchmarks to determine if the human is underloaded, overloaded, or “optimally” loaded.HARMONY relies on advanced physiological and psychophysiological sensing to track signals that past studies have shown correlate well with workload. A classification algorithm that uses deep learning methods learns the relationship between these signals, subjective workload assessments, and objective workload measures. Task-specific considerations, performance metrics, and environmental factors are additional inputs to the classification algorithm that ensure a high level of accuracy over the wide range of operational scenarios.The Phase I program will develop a taxonomy for H/A teaming that categorizes all of the relevant aspects of the problem including operational scenarios, autonomy levels, physiological sensing, subjective workload assessments, task-workload relationships, and performance metrics. Candidate signals will be selected, and sensor hardware based on prior BAI designs will be finalized. Proof-of-concept testing with pre-existing data will position the team to begin system evaluation early in Phase II.The team envisions conducting multiple sets of experiments in Phase II, building up from desktop simulator testing to fixed-base simulator testing to in-flight testing. To achieve the ambitious test build-up planned for the Phase II program, Phase I will also lay the groundwork for human subject testing by assembling the necessary paperwork for both the internal UND IRB as well as the Navy IRB.