Development of a Psychometrically Validated Spatial Disorientation Skills Assessment Tool

Spatial Disorientation (SD) is a contributing factor in many fatal aircraft accidents, resulting in significant loss of life and equipment.  For example, an analysis of SD mishaps in the U.S. Air Force from 1993 to 2013 identified 72 SD mishaps at a cost of 101 lives, 65 aircraft, and 2.32 billion dollars.  Further, published statistics may understate the severity of the SD problem.  Some researchers argue that SD accidents are underreported due to a variety of factors, including a failure to properly recognize some types of visual illusions as SD.  A tool that effectively measures the critical knowledge, skills, and abilities (KSA) for combatting SD hazards will play an integral role in developing and deploying improved SD training that reduces aviation accident rates and the associated human and economic costs.  Under the Phase I research effort, the team developed a design and psychometric validation protocol for the SD Skills Assessment Tool (SD-SAT), a survey and observation tool for assessing KSA for SD hazard mitigation.  SD-SAT is intended to support a variety of use cases.  A near-term application will be evaluation of candidate SD training programs/tools for procurement purposes.  In this capacity, SD-SAT can be employed both before and after pilots complete candidate training programs to assess improvement as a result of those programs.  Other potential uses for SD-SAT include routine assessment of SD skills, which could be employed to make decisions such as the frequency of training that will be applied across the fleet, or to make decisions regarding training needs for individual pilots.  SD-SAT will include multiple component scales to cover the cognitive, affective, and psychomotor domains.  The proposed research effort will implement the SD-SAT tool designed in Phase I, and execute the psychometric validation protocol for SD-SAT that was also developed in Phase I.  The validation process will involve multiple human-participant tests and will focus initially on the use case of evaluating SD training programs for procurement purposes.  A set of SD tasks in a full-cockpit motion simulator will be developed to serve as the “gold-standard” benchmark of SD hazard mitigation performance, and an important component of the psychometric validation work will involve examining the relationship between performance measured by SD-SAT and the “gold-standard” benchmark.   Another important aspect of the validation approach will be comparing performance measured by SD-SAT before and after human participants complete an established SD training program.  Human participant testing will also support usability evaluations of SD-SAT, and feedback from these evaluations will support improvements to the interface design and workflow.  The outcome of the proposed effort will be a fully functional and psychometrically validated SD-SAT implementation.