LLM-based Analysis and Structuring of multi-modal C4 data with Explainable Reasoning from Trustworthy Adaptation and Generation

Vadum will develop LASERTAG (LLM-based Analysis and Structuring of multi-modal C4 data with Explainable Reasoning from Trustworthy Adaptation and Generation) a novel machine learning framework that will intelligently analyze, efficiently structure, and intuitively explain large volumes of unstructured multi-modal data within Command, Control, Communications, and Computer (C4) Systems. Using natural language explanations, the architecture will build Department of Defense (DoD) operator confidence through a comprehensive understanding of the model’s decision-making process for multi-modal C4 inputs. LASERTAG will aggregate and store structured results in a high-performance database and deliver insights to analysts through an easy-to-use graphical interface. Vadum has demonstrated feasibility for the core aspects of the LASERTAG solution on prior efforts and developed early-stage prototypes for several subsystems, which will be used to produce a TRL-6 solution by the end of Phase II. LASERTAG will modify an open-source large language model (LLM) backbone to process, understand, and give structure to multi-modal inputs such as audio, images, and text. The system preserves the context required for generating explanations for the system’s behavior and further improves its results by simultaneously generating high-quality explanations using novel automatic, intelligent prompt modification techniques. An efficient data ingestion and storage pipeline leverages an efficient database that allows LASTERTAG to process and structure large amounts of data while enabling retrieval of results, explanations, and reference data at scale. The automatic and rapid formation of knowledge graphs provides additional structure beyond the results of individual tasks or within singular database fields.  LASERTAG will deploy an intuitive front-end interface that enables DoD operators to interact with the system and its outputs with ease, in turn building confidence in its results and providing insight into the underlying mechanics of the generative process.