MASS-OP: An Open Platform for Managing Agentic Systems at Scale

This proposal focuses on leveraging LLMs in defense by creating agentic systems that can act as "reasoning engines," interpreting natural language problems with contextual information. We aim to model a U-28 aircrew, which includes a pilot, co-pilot, and combat support officer, conducting intelligence, surveillance, and reconnaissance (ISR) operations. The simulated crew will use a knowledge base from publicly available documents and tactics, techniques, and procedures (TTPs) along with MathWorks toolboxes for generating tactical behaviors.Ensuring the safety, security, and reliability of these agentic systems is paramount. We propose a testing process to evaluate agent responses to valid and invalid inputs, forming a model for testing agentic autonomous systems in modeling and simulation (M&S) exercises. To maximize extensibility, we will develop the framework as an open platform (MASS-OP) under an Apache 2.0 license, facilitating contributions and use by multiple organizations. MASS-OP will support programming agentic organizations, configuration management, and continuous monitoring, forming part of a "defense commons" to accelerate and de-risk the adoption of agentic autonomous systems across the Department of Defense.