Deep Media: Enhancing and Automating PAI Analysis Using the DubSynch Universal Translator Platform

The Israel-Gaza conflict underscores the major role open source intelligence (OSINT) and publicly available information (PAI) assumed in modern conflicts. OSINT is a key domain for recognizing adversary activities and potential threats. This includes adversary disinformation or psychological operations, indicators of military movements, the identities and whereabouts of key leaders and hostages, and local attitudes. The National Air and Space Intelligence Center (NASIC) fills a crucial role in the DAF/DoD by analyzing these domains through language audio/video transcription, translation, and advanced biometric capabilities in order to convert PAI into actionable OSINT. However, current language translation, transcription and analysis tools and processes are inaccurate, time-consuming, and unscalable. As a consequence, the time to gather and analyze PAI data and convert it into actionable OSINT is prolonged. In this time, enemy forces can move people and weapons into position for an attack, but DAF/DoD analysts are mired by cumbersome workflows that slow decision making.NASIC and the broader DAF intelligence community need an automated language translation and transcription system that increases accuracy and streamlines manual processes so the Air Force can capitalize on OSINT and gain a decision advantage against adversaries. To address this need, NASIC proposes to partner with DeepMedia, Inc. on an SBIR Phase II project. DeepMedia will modify and adapt its DubSync Universal Translation platform for NASIC mission requirements. DubSync is an advanced AI-driven translation tool designed for the non-Defense commercial market. It features large-scale transformer models and traditional Natural Language Processing (NLP) techniques, supporting English and 199 other languages, including rare ones, with a Cycle-Transformer No Language Left Behind (CTNLLB) model for enhanced accuracy and contextual understanding. The effort will enhance language detection capabilities, ensuring higher precision in automatic speech transcription. DeepMedia will re-engineer the underlying model architectures, enrich training datasets with real-world data augmentations, and fine-tune transcription and translation machine learning models to resonate with AF-end user specifications. Furthermore, DeepMedia will modify its face and voice recognition technology, enabling accurate identification of individuals across diverse audio/video media. This will include advanced facial feature mapping, voice isolation, and language-agnostic speaker recognition, ensuring an unmatched accuracy level and context-sensitive identifications. The custom-built solution for NASIC will encompass the development of new, AF-centric recognition models. These will be integrated with DeepMedia’s expansive face and voice data repository, enriched with real-world augmentations during the training phase, and will employ sophisticated ML architectures.