Generative ATC AI Plug-In Integration for Flight Simulators and VR/XR Devices

Effective and accelerated production of pilots is a key need across the USAF. Increasing access to training content and relying on advanced immersive training technologies has been a persistent goal to provide an effective and low-cost training option . These currently lack the realism of air traffic control and voice-interaction with synthetic entities to provide a truly realistic training environment. Augmenting the capabilities of flight training devices to include radio communication, voice-based interaction, and real-time feedback on performance and proficiency brings enhanced realism to training and shortens the knowledge acquisition cycle, delivering higher quality training and increasing the training pipeline capacity.We propose delivering a generative artificial intelligence air traffic control capability (ATC AI) and adapting the standalone ARSim (aviation radio simulator) into a plug-in platform for flight simulators, extended reality (XR), and virtual reality (VR) training devices. The effort directly supports the critical technology areas of advanced computing and software and of human-machine interfaces. It provides a new capability for simulators and XR/VR training devices creating highly immersive realistic training environments with immediate feedback. Development and integration of intelligent ATC agents, built on a combination of large language models and proprietary knowledge graphs, add to flight training a level of realism, flexibility, and scalability currently not available in the marketplace. The new technology enables the USAF to enhance flight training, accelerate pilot production, and improve the overall quality of training with cutting edge technology and a scalable and low-cost architecture. This capability will enable pilots to interact with ATC AI agents in all phases of flight, in civilian and military airspaces, and will enable instructors to dynamically inject disruptions and modifications during training sessions. The proven speech and performance assessment will provide instructors objective and consistent performance evaluation.The ATC AI capability has been demonstrated for civilian use in collaboration with flight training institutions and simulator providers. We have proven the ability to control multiple aircraft in departure and approach phases of flight, to interact with human pilots to issue instructions and clearances, to re-evaluate airspace control goals based on unforeseen pilot actions, and to deconflict aircraft operating in the same airspace. The proposed solution has been integrated with XPlane11/12 to provide immediate feedback on both radio communication accuracy and proficiency as well as procedural knowledge and execution. The proven technology readiness and interest and partnerships with flight training institutions significantly increase the likelihood of transition success, demonstrates market interest, and considerably reduce the risk to the USAF for this Phase II effort.