Multimodal Interface for Natural Operator Teaming with Autonomous Robots (MINOTAUR)

The Armed Forces have a clear need for unmanned ground vehicles (UGV) that can autonomously accompany a soldier or vehicle during maneuvers through complex environments. Such UGVs will help solve logistical problems such as individual soldiers needing to transport more equipment and supplies than they can carry in a backpack. While various semi-autonomous leader-follower prototypes have been developed in recent years, current systems remain unreliable, cumbersome, and unable to adapt, trading off an operators physical burden for an increased cognitive workload, and eroding the operators trust in the system. The team of Charles River Analytics and 5D Robotics proposes to develop a Multi-modal Interface for Natural Operator Teaming with Autonomous Robots (MINOTAUR). MINOTAUR fuses multiple proven leader-tracking and robot control technologies to provide a natural and reliable hands-free soldier-machine interface (SMI) for soldiers operating in challenging environments. In particular, the system fuses state-of-the-art camera (EO/IR), 2D LIDAR, and ultra-wide band (UWB) beacon based leader-following technologies to eliminate typical failure modes in leader following solutions. The system fuses context cues with inputs from multiple control modalities (e.g., voice and gesture commands) to infer the operators intent in ambiguous situations and provide an appropriate level of robot feedback.