Adaptive Augmented Reality Eabled Electronic Procedure Toolset

The proposed research is aimed at investigating the feasibility to provide an integrated tool suite for development of Adaptive user interfaces for Augmented Reality (AR) enabled electronic Procedures. Human Space program has relied on procedures to operate efficiently and safely any spacecraft systems since its inception. Over the past few years, efforts to enhance these procedures using Augmented Reality has demonstrated the possibility to increase crew autonomy from ground support by providing improved guidance and just in time training. Many challenges persist in the following areas: difficult and time consuming development of the AR material, a range of potential AR hardware devices and software platforms to deploy on, and limited flexibility of the User Interface causing underwhelming user experience. Tietronix proposes to develop a toolset that will support the development of robust and adaptive user interface for the electronic procedures that leverages the power of AR technology. The envisioned toolset combines an authoring environment that enables the user to create electronic procedures with adaptive user interfaces which are dependent on their context of-use defined in terms of the user, platform, and environment. The UI can adapt to multiple elements such as the context in which the procedure is executed (workload, stressful conditions), the user skill level, the deployment platform (AR headset type, VR environment, tablet, phone), the type of cues to be provided. The use of this toolset will enable procedure developers to provide to operators enhanced situational awareness during the execution of the procedures by overlaying additional information such as instructions or graphical cues on top of the target system views, and auditory or tactile inputs. The proposed integrated suite of tools will provide critical technologies needed to develop adaptive UI for AR enabled electronic procedures, and execute these within the context of NASA space program.