Exploiting Unobtrusive Psychophysiological Sensors and Work Context to Enhance Monitoring of Human-Automation Interaction

We propose a three-pronged approach to advancing the state of the art in human state assessment. First, we will employ a small number of complementary sensors that can be naturally integrated into the work setting without complicated setup requirements. By integrating unobtrusively into the operator"s environment, we hope to minimize the artificiality that can impact operator behavior. Second, in order to successfully connect the collected mental state data to the human-automation interaction, our system will analyze that data in the context of data describing the operator"s attention dynamics and physical performance as well as the status of the automation involved. Finally, we will employ a unified probabilistic framework that will allow the proposed system to fuse the disparate data sources in a manner that: exploits their partial redundancy to mitigate the effects of data artifacts and missing values; appreciates the temporal nature of the data to allow for the analysis of trends rather than absolute values; and enables simultaneous reasoning bottom-up (from data) and top-down (task-based hypotheses). Our Phase I research and prototyping will result in a proof-of-concept prototype with sufficient depth to set the stage for the complete implementation of a human state assessment framework in Phase II.