Real-Time Detection of Operator Workload as Input to Scalable Autonomy During Dynamic Mission Operations

As the Department of Defense continues to modernize its forces and capabilities to meet and overcome next-generation threats, advanced automation enabling technologies such as machine learning (ML) and artificial intelligence (AI) will be a part of the solution. The question of how AI applications integrate with the operator is important. This includes not only what the AI will do to aid the human but also when that aid is enabled. The “what” portion of that question will be as varied as the role of the Naval aviator and the requirements for a given mission. The question of when aid is enabled / disabled and how is equally complex given the range of potential automation technologies and their relative scale of influence. Too soon, and the airman may view the assistance as a nuisance or distraction, negatively impacting the acceptance of the automation and perhaps the overall mission performance. Too late, and the aid may no longer be relevant to the changing environment. Delayed aids may also result in prolonged and elevated workload on the airman which may contribute to a cumulative effect on overall fatigue and reserved attentional resources. In addition, delays in enabling the augmentation may allow a situation to deteriorate further and place an increased burden on either the airman, the automation, or both. The cascading effects of this workload on the crew must also be considered, multi-crew aircraft cannot consider individual airman workload levels in isolation. To address the issue of how and when to trigger this aid, a team led by Systems Technology, Inc. proposes to develop the Autonomous Domain Agnostic Physiology for Automation Tasking (ADAPT) system, a novel suite of physiological measures/sensors, auditory channels, relevant aircraft information, and advanced identification and filtering techniques that provides individual and team workload levels, enabling future workload reducing adaptive AI triggers.The ADAPT system is a combined hardware/software solution designed to collect physiological measures and Naval aircrew activity, and classify both the individual level and overall team workload state to determine if AI assistance is necessary to maintain the required level of task performance and workload levels. If assistance is required, the ADAPT system will provide an electronic trigger to an external (to be defined) AI/ML system to provide appropriate assistance to the operator/team. If “no”, the system will do nothing or perhaps return some responsibilities back to the crew to ensure they maintain an appropriate level of engagement. ADAPT will build on STI’s Pilot Assessment Scaling System (PASS), the only technology currently capable of predicting test pilot feedback and ratings for handling qualities evaluations. This has been sought after for decades and PASS offers the first demonstrated capability to do so, leveraging artificial neural networks and synthetic data generation to optimally use an otherwise limited test data.