Weapon System Operator Multimedia Tactical Operation Aids

Relatively inexpensive human interface peripheral devices/software, such as natural speech recognition software, gesture recognition data gloves, wireless mice, head-mounted displays, and eye tracking systems, are now commercially available. Additionally,there is increasing interest in the development of future military command and control systems for human assistance by intelligent agents (in the form of agent associate surrogates) to make human operators more efficient and decrease the cognitive burdenneeded for real-time tactical decision making. ORINCON is a leader in advanced agent-based intelligent user interface prototype developments for future naval platforms such as DD-X. The integration of multiple human interface modalities with an advancedagent associate framework can provide a robust and flexible intelligent multimodal interface to increase efficiencies of weapon system operators working in a variety of command and control environments like the E2-C2. By using the existing workstationagent-based intelligent user interface automation framework being developed for the DD-X combat system, along with COTS multimodal interface devices as a starting point for this effort. The risk to accomplish program objectives is reduced while theimmediate benefit to the Navy is maximized. This Phase I SBIR effort will provide a demonstrable foundation for a Phase II effort to develop a comprehensive intelligent user interface (IUI) workstation prototype. This prototype will integrate multiplehuman interface modalities with an agent associate architecture that can be applied as an agent-mediated intelligent multimedia interface to increase operator efficiencies in performing E-2C tactical command and control tasks. The resultant IUI workstationdeveloped under Phase II will be tested and integrated in a Navy E-2C laboratory environment such as the E-2C System Test and Evaluation Laboratory (ESTEL). Tactical command and control scenarios relevant to E2C missions will be developed and measures ofincreased operator efficiencies will be determined. These MOEs will include E2C-specific mission effectiveness measures as well as generic operator effectiveness response measures currently being used to evaluate advanced operator workstation concepts forthe DD-X platform. The development and integration of a domain-independent agent associate architecture with advanced COTS multimodal interface devices can be applied to numerous military and commercial domains that require an intelligent interactivecontrol of complex systems by human operators. Examples are power plant operations, space station operations, and advanced military C4ISR systems.