Speech Interface Architectures for Human Interaction with Computer-Generated Forces

"Combat superiority requires training superiority. Current training state-of-the-art combines virtual reality, gaming techniques and intelligent forces. A key element missing from this equation is realistic speech interaction, with intelligent agentsrepresenting opposing forces, friendly forces and non-combatants. This effort will combine current research in speech interaction and understanding with proven technology for communicating with intelligent agents, to produce a more realistic trainingenvironment for warfighters.Communication between agents, humans and agents, and intelligent agents and other constructive forces, must be robust and sufficiently natural for human interaction, account for the inherent ambiguity of spoken language, and allow for processing timedelays and multi-person communications with interruptions.To more readily enable interchange of roles between humans and intelligent agents, this research will model mechanisms used in the real world rather than create simulation-specific versions. For example, radio messages are natural language representationsof the same, doctrinally correct, English utterances spoken by Marine and Navy warfighters and commanders.We will survey speech interaction research (both human/human and human-agent), explore relevant issues related to realism for virtual training, evaluate speech interaction networking architectures, and propose a plan for transferring the next generation inspeech research into a key component of effective military training. Expected benefits will be greater realism and believability (for example, via emotion-enhanced speech patterns, inflection and intonation), improved management of multi-person/agent communications (including cross-talk and interruptions, processing timedelays, and multiple language support), better speaker recognition and flexibility in deviating from strictly specified grammar, and intelligent agents that are more adept at communications (knowing how and when to answer, when to repeat themselves orrephrase, gain understanding despite interrupti