Extracting the Emotional and Attentive State of Students Using a Real-Time Visual Sensor

WaveBand will develop an advanced facial expression analysis solution implemented as a real-time system to successfully measure a student's emotional state with the use of low-cost web cameras. The system uses that information to modulate the reactions ofan intelligent tutoring system. The proposed solution goes beyond the state-of-the art by interpreting facial motion data obtained from a facial feature tracker in the context of psychological and educational content. This context-based approach offersmajor advantages over the passive interpretation of facial communication cues as it eliminates invalid combinations at an early stage and thereby increases recognition accuracy. Furthermore, it naturally lends itself to the interpretation of characteristictrainee behavior, such as attention, mental effort, hesitation or confusion, and fatigue, as they relate to the progress of a learning session in the computer-based training system.The system called, AffectCam, analyzes the students' emotional and attentive state in the context of a multi-modal dialogue model in which the learner is treated as part of a collaborative process and non-verbal cuesare interpreted in relation to theevents of the learning session itself. This active approach offers critical advantages over existing Intelligent Tutoring Systems, as it mimics the process of face-to-face human interaction.