HMD-Compatible Mission Performance Measurement System and Tools

ABSTRACT: Helmet Mounted Displays (HMDs) will soon replace the Heads-Up Displays (HUDs) found in many tactical combat aircraft, beginning with the F-35, the first 5th generation combat aircraft to exclusively use HMDs as the primary instrument and sensor display. Currently, it is difficult to objectively measure operator performance and to capture the interaction between the human operator and the HMD, such as what information is being attended to. Thus, there is an opportunity to utilize advances in physiological performance monitoring technologies to develop practical assessment tools with associated instrumentation to better quantify real-time assessment and performance monitoring of pilot performance and improve feedback via real-time intervention and enhanced after action reviews (AARs)/debriefings for more effective training and assessments of pilot proficiency. The proposed HMD ASSESS system will incorporate a suite of measurement tools, including cognitive state (EEG), behavioral metrics (eye tracking), and system metrics to diagnose root cause(s) of operator performance deficiencies/inefficiencies and provide individualized, targeted feedback. HMD ASSESS will provide revolutionary performance monitoring capabilities to optimize assessments of pilot proficiency and training effectiveness and efficiency by providing next-generation neurophysiology-driven, individualized understanding pilot interactions within a cockpit using an HMD. BENEFIT: The proposed HMD ASSESS system will incorporate brain-based and physiological sensor technology with traditional behavioral and system-collected metrics to better understand what information is presented in the HMD, and how a pilot interacts with presented data including reactions and actions taken based on the data provided. This comprehensive, objective understanding of pilot interactions will be used to measure and diagnose pilot deficiencies/inefficiencies in real-time, and can be used dynamically monitor pilot performance, provide training evaluations and assessments of pilot proficiency, and populate targeted AARs/debriefings to enhance understanding of pilot errors and related root cause(s). Such an innovative measurement and diagnostic solution will substantially improve pilot performance assessment and initial and sustainment training by quantitatively evaluating visual scan and search tasks via HMD-presented information and information gained from out the window views. Such innovative metrics will peer into the"mind of the pilots", thereby providing the capability to monitor traditionally"unobservable"performance related to perceptual and cognitive aspects of piloting. HMD ASSESS will be one of the first real-time, closed-loop assessment systems that incorporates considerate adaptations as well as AAR feedback driven by effectively diagnosed training deficiencies/inefficiencies and root cause analysis of error patterns. Build on individual results, HMD ASSESS will provide targeted feedback in real-time to optimize performance and/or training opportunities. Because HMD ASSESS will be a stand-alone system, it may be used to provide training in the absence of instructors (particularly for sustainment training) by tracking and recording performance, training deficiencies, and guiding targeted training feedback to optimize performance, thereby enhancing time spent training (as opposed to simply practicing through repeat, unmeasured exposure) tasks while not increasing the load on instructors.