Visual Intelligence for Situation Awareness

A key challenge in remote robot operations is the need to maintain situation awareness during operations. Current LEO ISS operations require operators to manually manage multiple 2D cameras on station and EVA suits to obtain views of the operating environment and demands operators perform mental 3D reconstruction using these views. Remote ground operators face these challenges compounded by restrictions on bandwidth and limited downlinks that require prioritizing camera feeds to downlink. Compared to current LEO operations, it is anticipated that lunar LTV and PR operations will have decreased available bandwidth and downlinks, which will exacerbate these issues. To address this, TRACLabs proposes Visual Intelligence for SiTuation Awareness (VISTA). VISTA will intelligently select RGB cameras in the remote operating area that provide task-relevant context for current operations. Selected camera feeds will be downlinked and processed on the ground with photogrammetry to provide interactive 3D views of the operating area, allowing for effective monitoring or control of remote systems. Phase I funding will allow the development of a proof-of-concept VISTA system. TRACLabs plans to target Commercial Lunar Payload Services (CLPS) providers and government organizations as targeted markets.