Multi-Domain Data Fusion Instructional Strategies and Methods for Pilot Training

In October 2020, HF Designworks, Inc. (HFDW) was awarded a NAVAIR Phase I Small Business Innovative Research (SBIR) contract titled “Multi-Domain Data Fusion Instructional Strategies and Methods for Pilot Training.” During this Phase I effort the performers created a Learning Management System (LMS) introducing Artificial Intelligence (AI) and Multi-Domain Data-Fusion (MDDF) technologies that help pilots to maintain Situational Awareness (SA). Our desire for the Phase I effort was to create a Navy Future Vertical Lift (FVL) specific Concept of Operations (CONOPS) with the anticipated Future Long-Range Assault Aircraft (FLRAA) platform and combine our backgrounds in human performance modeling, User Experience (UX) design, technology integration, LMS architecture/systems, and Artificial Intelligence (AI) to build a training tool for pilots where they could view Multi-Domain Data Fusion (MDDF) from various sensors including FVL’s Air Launched Effects (ALEs).   The key goals of our work are to teach the pilot to: 1) understand and trust the information that is being fused and presented, and 2) better understand how to work with Artificial Intelligence (AI) and autonomous systems to achieve better Situational Awareness (SA) and combat effectiveness. Our platform drives continual and iterative improvement of autonomous systems, as well as overall human/AI teaming success as we move toward the launch of FVL platforms.  During our Phase I effort, we constructed a CONOPS around an Integrated Air Defense System (IADS) Breach mission where two FLRAA were tasked with entering into a near-peer adversarial city with known advanced Surface-to-Air Missiles (SAM) to drop off a Sea, Air, and Land (SEAL) team. The SEAL team was tasked with looking for a High Value Target (HVT). Our human performance models of FARA missions and now FLRAA missions confirmed that areas of highest workload for such a mission type were when the pilot (for single crew missions) or the copilot (in missions where we modeled a two-person crew) had to perform their traditional tasks as well as supervise the ALEs.  Recognizing these periods of highest workload, we created our Phase I LMS around the operation of ALEs to include the data feeds they provide and the support of AI which would be needed to bring workload down to acceptable (i.e., manageable) levels. While our initial Phase I LMS walks the pilot through a portion of the mission, our Phase II goal is to develop an LMS that provides preparation - and pilot performance measurement - throughout all phases of the mission and pilot training, and - critically – introduces the ability to progressively train and improve the AI/autonomous capabilities which support the pilot. In the course of Phase II we will also automate mission creation tools, enabling us to quickly support the simulation of a variety of missions, platforms, and user types, from a student Naval Aviator to combat pilot.