A Novel Non-Intrusive Approach to Detect Human Fatigue in Real-time

Fatigue affects human performance. If we can capture the early signs of fatigue such as lack of concentration, yawning, changes in voice characteristics, etc., we will be able to evaluate individual job performance and plan optimal work schedules to optimize performance. Intelligent Automation, Incorporated (IAI) and its subcontractor, Prof. Jianbo Shi and Prof. David Dinges of the University of Pennsylvania, propose a novel non-intrusive approach to detecting human fatigue. Our approach combines the latest algorithms in video processing, audio processing, and fusion in a unified framework. The video signals contain not only camera signals but also eye safe low intensity laser scanner signals (3-D images) that can provide more facial information than 2-D images. Our video processing algorithms extract coarse (body behavior such as yawning) and fine (eye blink, facial expressions, etc.) features. The microphone signals contain voice characteristics that reflect fatigue behavior. Our audio algorithm is based on cepstral features and Gaussian Mixture Model (GMM), which have been proven to be extremely powerful in characterizing human speaker and bird voice characteristics. Our fusion algorithm combines video and audio fatigue features and yields an optimal decision on fatigue detection.