Librarian - AI Driven Multi-Int Unifying Platform Software Tool

The number of sources and amount of data that Naval intelligence analysts are required to manually sift through in order to ensure maritime forces have both actionable intel and provide decision advantage for their commanders is daunting. Today, existing tools are time-consuming, workforce intensive, and cumbersome to process and distribute in a timely fashion. Fortunately, this is a problem that can be addressed using modern machine learning (ML). Mosaic ATM, Inc. is proposing the development of a software tool called Librarian, that a) accepts, parses, and stores multi-modal data sources; b) annotates the data source metadata; c) employs a collection of transformer-based neural network models that annotate and represent the content of the data as natural language or in an embedding structure that is common across all modes to allow cross-modality search; and d) exposes a user interface (UI) or application programming interface (API) that lets analysts easily query data across modalities. Mosaic's Phase I technical approach will be to first implement a solution using pretrained models to perform their native task or a related task using zero-shot inference. Example pretrained models that could be utilized include BERT-(language), CLIP-(visual-language), GPT-3-(language), and DETR-(visual-language and object detection). Second, Mosaic will enhance the performance of the innovation by incorporating domain-specific data. For language models, Mosaic will explore a self-supervised method called Generative Pseudo-Labeling (GPL). Mosaic will also explore an image model, specifically a technique called SimCLR. The third and final step will be advanced multi-modal inference which will be approached as a causal language modeling (CLM) problem, i.e., text generation. OpenAI GPT-3 is the current state of the art for such models and Mosaic will also explore a more cost-efficient variant of GPT-3, GPT-J or GPT-2 for fine tuning. The three scenarios that Mosaic proposes to utilize for ML algorithm development and model training are: Scenario #1) Intelligence, surveillance, and reconnaissance (ISR), protection, and defense of a key international maritime port utilized by commercial and military vessels of various nations; Scenario #2) Support of local LEAs conducting security at a high-risk event such as the Boston Marathon; and Scenario #3) Intelligence support of a Carrier Strike Group conducting an international strait transit such as through the Strait of Hormuz. As a means to bring capability to the end users in the most efficient and timely manner, Ultra has agreed to team with Mosaic for Phase II technology development and Phase III commercialization and U.S. Navy implementation. Ultra's Situational Awareness Management System (SAMS) is a proven ISR platform that is perfectly suited to host future Mosaic technology that will solve the Navy's, and perhaps DoD, challenges with respect to intelligence analysis efficiency and performance.