Voice Communication Assessment for Integrated Training System using Speech Recognition and Natural Language Processing

This Small Business Innovation Research Phase I project will define and develop a concept and prototype design for a stand-alone, Voice Assessment System (VAS) for quantitative analysis of cross-platform communications collected during E-2D Air Defense simulation training exercises. The VAS will apply state-of-the-art Automatic Speech Recognition (ASR), Natural Language Processing (NLP), and Artificial Intelligence (AI) to provide quantitative analysis tools to help instructors assess the communications and overall performance and decision-making effectiveness of a mission. The VAS will go beyond simple transcriptions of “what was said” by providing various context-specific analytics to correlate the voice communication with tactical execution for the instructors to provide relevant feedback. Using NLP, it will be able to track and map the mission’s voice communication among the officers, pilots, and shipboard Commander. The VAS will transcribe and time-stamp the voice recordings and produce a meta-tagged and parseable “utterance log” from multiple participants. This utterance log will support the instructors with their assessment and will integrate with the Next Generation Threat System (NGTS) Analysis and Reporting Tool (ART) system.  Using the VAS meta-tagged transcriptions, analysts will be able to locate specific actions that occurred during the training scenario and highlight participants’ actions, response times, and decisions. Currently, instructors use manual and qualitative techniques to analyze and correlate the voice communications with other synthetic data logged during the multi associated training scenario and identify areas that need improvement. Using advanced speech technology, the VAS will support assessment automation and help instructors in the following areas: Decrease instructor workload Reduce human error Reduce manpower time requirements Automatically track and map communications to provide information on protocol adherence and timeliness The overall goal of this project is to develop a Voice Assessment System and capability to help accurately measure and analyze task performance at various levels, including individuals, teams, multi-teams, and organizations, and to enhance warfighter performance and effectiveness.