Innovative Hands-Free Point-and-Click Computer Control Device for Vehicles and Objective Force Warrior

Soldier performance in a modern battlefield environment is becoming increasingly demanding on the human sensory facilities and more lethal due to threat of biological, chemical, and other agents. The logical defense mechanism is to encapsulate and try to isolate the soldier from the hostile environment. However, when the human head is enveloped in a protective sealed helmet or the soldier is confined to a sealed vehicle; he becomes disoriented because of the denial or mismatch of ordinary sensor cues. The resultant effect of this disorientation causes the soldier to lose the vital situational awareness and the feeling of well-being needed to react to changing threats and to carryout his mission. The Point and Click/Speech (PAC/S) research opportunity is a solution to this and other human computer interface problems while maintaining local surrounding audio cue orientation and situational awareness. Unique TERI Natural Language Speech, integrated with enhanced speech recognition and language understanding, is to be seamlessly incorporated with tactical operation aids, 3-D audio cueing and eye-tracking, providing a cooperative modality that is user friendly and intuitive. Universal interface standards will be integrated among the PAC/S components, with application to Objective Force Warrior, Homeland Security Architecture, and Combat Decision Aid System, to offer an Army Technical Architecture and commercial compliant solution. The integrated PAC/S performance will be demonstrated using a "plug-in" architecture by a soldier to test OFW portable Multi-Modal Interface (MMI) suitability and to quantify benefits. Incremental deliveries of the enhanced MMI components are scheduled for delivery every six months. Tactical speech recognition research at TERI has made great strides in the past few years in terms of noise robustness and language processing, and is gaining acceptance as a keyboard replacement. A hands-free mouse (trackball) replacement that can be operated accurately in a moving vehicle or by a soldier in the field, will enable the warrior to navigate through menus, select items, and control C3 information technology components. A MMI low power lightweight PAC/S device will be delivered, providing unique tactical operation aids for an intuitive and natural feel. Operator training will be minimized, and reaction time will be improved. OFW soldiers using this MMI device will gain increased lethality and improved survivability.