A System to Analyze Facial Features to Enable Operator Condition Tracking (AFFECT)

The duties of modern military personnel often require the completion of challenging tasks, in stressful environments, and with multiple demands competing for the warfighter’s focus. Measuring the stress, anxiety, uncertainty, and fatigue (SAUF) of the warfighter during a particular task would have multiple benefits: (1) commanders would be able to detect problems and reallocate workload or address uncertainties more effectively; (2) work products could be judged by the conditions under which they were developed; and (3) systems could be developed to automatically or semi-automatically mitigate SAUF. However, in the current state of the art in SAUF identification, individual indicators are considered in isolation across feature space and time. To address the need for non-contact classification of stress, anxiety, uncertainty, and fatigue in interface applications, we propose to design and demonstrate a system to Analyze Facial Features to Enable Operator Condition Tracking (AFFECT). The AFFECT system combines powerful classification techniques with a multi-dimensional, temporal data model of novel visible and thermal features to enable automatic, non-invasive detection of SAUF conditions in a subject. The development of this technology will in turn enable dramatic improvements to the training, workflow, and overall effectiveness of the warfighter and analysts throughout the DoD.