Sub-centimeter Resolution Lunar Digital Twin for Collaborative and Immersive Mission Planning

Alphacore and Arizona State University (ASU) propose the development of hardware and software solutions for digital twins of regions of strategic interest on the moon, for robotic mission planning in preparation for and potentially during ARTEMIS human missions. We are developing a next generation imaging system for terrain mapping in dynamic lighting and thermal conditions. When deployed aboard rovers, such 3D imaging systems will produce multi-modal, high resolution imagery that can facilitate AR/VR enabled exploration of terrain at sub-cm ground sampling resolution. In this immersive digital twin, astronauts and stakeholders can carry out assessment and visioning activities as well as mission drills. We identify 1 mm/pixel as an ideal target resolution for terrain assessment, with sub-centimeter scale being the milestone for this proposal, with potential for millimeter scale in phase II if phase I is successful. We will leverage ML algorithms such as diffusion models and neural radiance fields (NeRF), for view synthesis from sparse imagery. Impact of generative features in astronaut mission planning will be assessed on earth analog sites in Arizona. Leveraging these sites, we will develop our methods focusing on synthesis of photorealistic and interactive geological features such as rocks, soil, lava tubes, and craters. Apollo astronaut imagery and ASU LROC imagery will be leveraged extensively.Our goal for phase I is to provide hardware solutions and a software pipeline for high-resolution terrain assessment (sub-cm resolution) with semantic and geologic trait metadata estimated by humans and AI. 3D rock models from photogrammetry, lidar scans, and spectroscopy will be assimilated to synthesize arbitrary views for multiple participants, enabling use in rendering engines at NASA (Unreal and Unity). Finally, this development will support also the creation of highly immersive and high-accuracy simulation environments for rover operations.