Context-Sensitive Augmented Reality for Mission Operations

Current NASA missions to the International Space Station (ISS) are heavily dependent upon ground controllers to assist crew members in performing routine operations and maintenance as well as responses to off-nominal situations. Performing these procedures often requires close collaboration between ground controllers who have deep knowledge of the spacecraft's systems. This collaboration becomes more difficult in extended missions and crew members will need to have more autonomy. Augmented and virtual reality technology can help replace some of the guidance that ground controllers offer to crew members during procedure execution. Our proposed approach focuses on integrating augmented and virtual reality technologies into the same tools that mission operations uses today, allowing for augmented and virtual reality assistance to be built and updated by flight controllers and other mission specialists just as they do for their other work products. In Phase I, TRACLabs integrated its PRIDE electronic procedure platform with augmented and virtual reality systems such as the HTC Vive. In Phase II we will extend the PRIDE platform to augmented reality devices such as the Microsoft HoloLens. TRACLabs is also partnered with the Georgia Institute of Technology's Augmented Environments Laboratory who will develop algorithms that dynamically adjust the augmented reality cues (both their position, orientation, and scale, along with the specific graphical elements used) such that they align with the necessary parts of the world without obscuring important parts of the world.