Multimodal Interaction Technologies to Support Small Unit Leaders

Unmanned systems can provide a wealth of information to the warfighter on the modern battlefield, and there are a multitude of control modalities available to the operator. Each modality lends itself to different tactical situations and the dynamic selection of the correct modality can optimize both mission performance and situational awareness. BANC3 is a leader in the development of Augmented Reality headsets for the commercial and DoD market. The company has significant experience in the design, development and deployment of lightweight HUDs, in house speech recognition expertise, head tracking, eye tracking, gesture recognition, and positional tracking. BANC3 proposes to use our open architecture Android Augmented Reality headset to develop a demonstration system that automatically switches control of a COTS unmanned drone from a tablet, to a headset HUD with dynamic switching between eye tracking control, speech and gesture control. We will support a scenario where the warfighter begins by controlling the drone with a tablet, then automatically switches to hands free HUD control and back again effortlessly.