Augmented Reality for Maintenance of Aircraft with Ecological Displays and Artificial Intelligence-based Assistant (ARMADA)

Navy maintenance personnel are faced with the critical tasks of diagnosing, repairing, and inspecting Navy hardware and equipment, including Navy aircraft assets. Successful maintenance is critical to the effectiveness, readiness, and safety of all Navy personnel, and reduces the costs of operation. Throughout the maintenance process, maintenance personnel make complex decisions regarding assessment of naval aircraft, prioritization of maintenance tasks, and how to best resolve a particular issue given part availability, personnel skills, and operational needs. To address these challenges, we propose to design and demonstrate an Augmented Reality for Maintenance of Aircraft with Ecological Displays and Artificial Intelligence-based Assistant (ARMADA) system. ARMADA will enable maintenance personnel to efficiently, flexibly, and effectively maintain aircraft by providing easy access to tailored information in real time through a multimodal, context-aware personal assistant support system. ARMADA will use a Cognitive Systems Engineering (CSE) approach combined with ecological, multimodal augmented reality (AR) interfaces and displays as well as a context-aware, natural language processing (e.g., a systemic functional grammar (SFG)-based approach) personal assistant to address the needs of maintenance personnel while they are executing maintenance tasks and procedures.