Equipment state recognition for augmented reality using CAD models

The goal for this project is to automatically recognize equipment relative to an Augmented Reality (AR) headset. This will allow holographic information to reference the equipment as the user sees it, without the need for fiducial markers on the equipment. Such holographic information is widely seen as a way to improve maintenance-action success rate and repair time. In Phase I, Overlab developed a prototype that estimated equipment pose relative to an AR headset by comparing sensor data from the headset to a previously provided 3D model of the equipment. The goal for Phase II is to generalize the Phase I solution. Presently, we assume that the equipment is completely static and that the environment is moderately illuminated. We propose to eliminate these restrictions by viewing the problem more broadly: we want to recognize equipment state using expanded a priori information about the equipment, and to do this with limited sensor data. The state information will be more broad than just the pose of the equipment. It will include estimating velocities for moving equipment and the configuration of equipment with detachable or moving parts. This state information is critical for augmented reality guidance; we cannot correctly place holograms if state information is unknown.