AI/ML Foreign Language Translation Integration with Automatic Speech Recognition Technology

The 17 TRW/517 TRG is seeking to improve foreign language proficiency testing of USAF linguists before they deploy to operational assignments.  According to a proficiency assessment expert on site at the Defense Language Institute (DLI), “the operational stakeholders demand a level of consistency and validity that right now we are unable to provide.” The 17 TRW/517 TRG has been unable to solve this enduring problem because conducting a holistic assessment or trend analysis of the Oral Proficiency Interviews (OPIs) used to validate linguist proficiency has been impossible to date since they are administered in 163 foreign languages.  This makes it impossible for even the best foreign language testing experts to conduct a comprehensive analysis across multiple languages for these tests.  To conduct this analysis--and thereby modernize the foreign language proficiency testing process for the thousands of airmen trainees detailed to learn a foreign language--the 17 TRW/517 TRG desires to translate a large sample of proficiency tests across many languages into English to enable a comprehensive follow-on analysis by language testing experts.  Attempting this task manually with the brute force of human labor alone would be prohibitively expensive--a technology solution is necessary for this work.  The persistent problem faced by the 17 TRW/517 TRG is the fact that the technology needed to perform this task does not exist today. This SBIR effort seeks to solve this longstanding problem for the 17 TRW/517 TRG by: Integrating Lilt’s Machine Translation (MT) capabilities with Automatic Speech Recognition (ASR) capability Adapting and customizing the applicable ASR and MT models to accommodate difficult-to-process speech forms, such as non-native speech Utilizing techniques such as data science and natural language processing to analyze the translated text There are four main objectives to this effort: Building an integration between Automatic Speech Recognition (ASR) and Machine Translation (MT) capabilities, which will benefit multiple DoD and Intelligence Community partners and stakeholders. Adapting and training the ASR and MT capabilities to accommodate difficult-to-process speech forms, such as non-native speech. Using the newly-developed capability to translate foreign language text, in this case foreign language student proficiency assessment interviews, into English, thereby enabling aggregate, data-based analysis. Developing an assessment framework (e.g. data science or natural language processing) to analyze the set of translated proficiency interviews in English, informing follow-on language training and proficiency assessment efforts across the USAF and other DoD services.