SENSEI – Sensing Emotion Non-invasively for Software-based Educational Instruction

DARPA and DOD often make use of electronic tutoring systems for administering additional training and ongoing education to their employees, creating an intimate one-on-one environment between the student and the electronic tutor. Unfortunately, even the most modern of these electronic tutors is unable to capture the emotional state of the student and thus cannot react to the student’s changing behavior. It has been shown that the most effective learning occurs when the instructor engages the student according to their current emotional state, and to that end UtopiaCompression (UC) is proposing a multi-modal intelligent tutoring system (ITS) that is capable of estimating the student’s emotional state using a number of external cues. UC’s proposed solution combines verbal, facial, and body posture cues through a novel probabilistic framework to accurately estimate the emotional state of the student. UC is also proposing a self-calibration procedure that tailors the system to the mannerisms of each specific student. The proposed technology can be deployed on platforms with a wide array of hardware capabilities. UC’s ITS will vastly improve the effectiveness of DARPA’s educational programs through accurate and efficient sensing of the student’s emotion, thus tailoring its program for the most interactive, lifelike experience possible.