Real Time Analysis and Fusion of Data from Imagers for Passive Characterization of Stress, Anxiety, Uncertainty and Fatigue

The purpose of this proposal is to present a novel and promising solution for real time detection of stress, anxiety, uncertainty and fatigue using passive features from thermal and visual videos of the face. Our solution consists of five modules: (1) data acquisition – using visual and thermal cameras capturing the face images in visible and thermal waveband. (2) Facial feature localization and tracking – including the eyes, eyebrows, nose, mouth and their spatial arrangement. With our sophisticated techniques, we can detect and track facial features from the face images with different facial expressions under various face orientations in real time. (3) Feature extraction and selection – based on the facial feature location, extracting the features that are related to SAUF, including expiration rate, heart rate, eyelid movement, head movement, etc. Our experience in emotion recognition and fatigue detection enables us to capture those physiological and behavioral features related to the psychological states. (4) Feature fusion and SAUF recognizer – we propose a dynamic statistical model that can monitor the change in each states and quantify its level. (5) Feedback – The computer can respond to an individual’s psychological change by either sending and alarm or offering necessary assistance.