Fast and Precise Automatic 3D Model Generation and Part Localization for Robotic Systems using AI and XR for Enhanced Maintenance and Repair

Fleet Readiness Centers (FRCs) are crucial for maintaining, repairing, and overhauling naval aircraft and components, which are essential for sustaining naval aviation readiness. However, these processes are often labor-intensive, hazardous, and inefficient, leading to longer turnaround times and potential risks to worker safety. The integration of advanced robotic technologies offers a solution by automating many industrial processes, thereby enhancing efficiency, safety, and productivity. However, there are significant challenges in integrating these advance robotic systems on various aircraft maintenance and repair tasks, such as painting, de-painting, blasting, grinding, disassembly, wielding, etc. For the robots to perform a designated task, the robot needs to have the following information:3D Digital Twin information – a CAD or accurate scanned model of the target object.Part Localization - Location and orientation of the target object.Keep out areas – location and “spatial” envelopes of other objects to avoid collisions and do robotic path planning.Typically, the 3D models of parts are not readily available (especially for legacy weapons systems) and the process of generating these models using dedicated laser or blue-light scanners require significant amount of time (hours to days) by a trained technical expert. Manual localization of the part(s) in the robotic booth – both the parts of interest and other objects – are fraught with human errors with significant risk of collision causing damage to the parts as well as the robot. Some robots have vision systems integrated with them – at a significantly increased cost – that can be used to scan the object and the 3D space to generate the 3D model as well as localize the parts in the booth. These robots, however, cannot be used in harsh environments, such as blasting, painting etc. These challenges were further confirmed through discussions with both commercial and DoD stakeholders, including NAVAIR FRC East. GridRaster’s Spatial AI and Dynamic Extended Reality solution for Robotics applications uses Computer Vision (CV), Artificial Intelligence (AI)/Machine Learning (ML), and Extended Reality (XR) to address all these challenges using only Commercial -off-the- shelf (COTS) XR devices and a local or remote server running GridRaster’s Spatial AI software. During the proposed Phase I effort, GridRaster will perform a comprehensive feasibility study within the NAVAIR FRC’s operational environment and demonstrate the capabilities and workflow of GridRaster’s system for robotics applications in an industrial setting. This Phase I effort establishes feasibility for a Phase II build-out of the proposed solution for Robotics applications to streamline various industrial processes, enhance worker safety, boost productivity, reduce turnaround times, and improve overall readiness.