Operationally Modulating Neurophysiological Interface for Extended Reality (OMNI Cog XR)

Astronauts encounter numerous cognitive and physiological challenges from the unique environment encountered during spaceflight. These challenges can significantly impact their health and performance throughout the duration of their missions. Cognitive challenges include altered sensory perceptions, disrupted circadian rhythms, and increased cognitive workload due to the demands of managing complex mission tasks confined spacecraft environment. Additionally, microgravity can lead to physiological alterations affecting astronauts' physical health and functioning. Artificial Intelligent Agents, monitoring these crewmember conditions and tailoring human computer interfaces, including XR platforms, might be utilized to better manage crewmember health an increase mission performance where operator decrements are detected. We propose the development of a portable, lightweight platform that can integrate cognitive/physiological monitoring sensors, and/or application-based measures of the same into an efficient data management and processing system (AI/ML) with dynamic two-way integration with XR systems (capturing camera or other data from HMD as well as sending real-time data to one or more HMDs). The innovative aspects of this system will be true multimodal data fusion (cognitive, physiological, and visual along with environmental telemetry) and analysis resulting in concise informational display and/or modulation of information presented to respective crewmembers based upon their KSA or operational state during task execution. Additionally, we propose the first “team” based XR applications, providing unique content and function depending on a specific crewmember’s role in tasks. To achieve this functional goal, we are partnering with medical and neurophysiological experts at UT Austin Dell Medical School to utilize their clinical expertise and existing, non-invasive cognitive/physiological monitoring platform funded by NIH and currently deployed in clinical trials.