Non-intrusive Hazardous Pilot Cognitive State Assessment via Semi-Supervised Deep Learning: CSA-Deep

In aviation history, many crew-related errors are caused by crew members being in hazardous cognitive states, such as overstress, disengagement, high fatigue, and ineffective crew coordination. To improve aviation safety, it is critical to monitor and predict hazardous cognitive states of crew members in a non-intrusive manner for designing mitigation strategies. In Next Generation Air Transportation System (NextGen) flight deck, emerging technologies will enable a transition from ground based navigation infrastructure to satellite based navigation and some control relating to separation of traffic will be delegated to the cockpit from Air Traffic Control (ATC). While the NextGen system will bring tremendous advantages in operational efficiency, the responsibilities of the pilot are expected to dramatically increase, which makes the hazardous cognitive state assessment even more critical.To address the above challenges, Intelligent Automation, Inc. (IAI), along with the Operator Performance Lab (OPL) in University of Iowa and Old Dominion University, proposes a real-time hazardous pilot Cognitive State Assessment system, called CSA-Deep, in all phases of flight for Integrated Crew-System Interaction (ICSI). The key innovation of the proposed research is the modeling and adaptive updating of hazardous cognitive states using a large amount of unlabeled data through semi-supervised deep learning.