Automation Support using non-Invasive Measures of Operator Vocalization (ASIMOV)

Human-machine teams are increasingly prevalent across the DoD. These teams unite human operators with advanced automated teammates to execute complex, mission-critical tasks. Unfortunately, this collaboration between automated and human teammates can increase operator strain, because operators must complete their own tasks while also monitoring teammates and distributing tasks; one specific strain occurs when an operator perceives an error and must make rapid decisions to complete task execution. A system that can detect when operators are experiencing significant strain (e.g., loss of confidence, frustration) and effectively provide solutions (e.g., offer assistance, automatically re-distribute tasks) to reduce or eliminate that strain can reduce operator workload, improve decision-making, and ultimately result in greater mission success. Such a system must monitor and assess operator state, have accurate indicators of operator strain, and predict how operator strain will impact task execution. To meet this goal, we propose to design and demonstrate a system for Autonomous Support using non-Invasive Measures of Operator Vocalization (ASIMOV). Under ASIMOV, we will research the relationships between physiological vocal and subvocal signals and operator strain, as well as how those indicators align with task execution to generate effective interventions that reduce operator strain and increase performance.