Verification, Validation, Assurance, and Trust of Machine Learning Models and Data

As the use of machine learning (ML) models proliferates in commercial and defense applications, the United States Army (Army) faces significant challenges in evaluating the effectiveness, robustness, and safety of these ML models in armament systems. For the Combat Capabilities Development Command (CCDC) Armaments Center, the decision to enable automated choices in these systems, which encompass lethality capabilities, requires very high confidence that any resultant behaviors will fall within intended operational and mission bounds. Ensuring reliable and safe behaviors involves both ensuring accurate and comprehensive test data is used in the creation and training of these ML systems and that the ML models are robust, accurate, and appropriately behaviorally bounded when employed against real data in practice.  ML models come in many forms, and the technologies used to create them are rapidly evolving. The Army needs 1) a process and framework to assess and measure the quality of training data and identify shortcomings that may lead to poorly trained ML models, and 2) a process and tools for ML model exploration that can assure confidence of model behavior within defined data boundaries and can also identify unintended or poor behavior in ML models if they exist. These processes and tools will need to be robust and flexible to handle various forms of ML models and data. OptTek Systems, Inc. (OptTek) and their research partner, the University of Alabama in Huntsville (UAH), are proposing to combine proven approaches and technologies into a set of processes and tools that can be applied to this problem along with an experienced team that has the background and expertise to achieve these goals. In this Phase I project, the OptTek team (OptTek and UAH) will explore a process for ML model data set validation and ML model behavioral evaluation that will help determine readiness for more formal operational test and evaluation.