A System to Analyze Facial Features to Enable Operator Condition Tracking (AFFECT)

Modern military and security scenarios increasingly demand rapid evaluation of human affective states, accurately detecting stress, anxiety, uncertainty, and fatigue (SAUF) to assess warfighter effectiveness or to advance security screening and interrogation objectives. In our Phase I effort, we designed and demonstrated a system to Analyze Facial Features to Enable Operator Condition Tracking (AFFECT) by extracting a broad array of features from visible and thermal spectrum imagery of the human face. We designed an effective stress induction protocol and recorded the resultant elevated stress levels using heart rate variability data. We demonstrated the feasibility of our classification methodology by implementing a prototype of the AFFECT system. For Phase II, we propose to expand on these results by designing new affective induction techniques and ground-truth metrics for anxiety, uncertainty, and fatigue. We will detect these states by applying novel visual and thermal feature extraction algorithms, and by implementing the temporal feature algorithms designed in Phase I. We are confident that the sound principles underlying our Phase I approach will provide a robust foundation for expanded SAUF classification in Phase II.