Model-Inspector: Detecting AI/ML Model Weaknesses via Bill-of-Materials-based Analysis

We propose Model-Inspector, an automated toolkit for creating, maintaining, and assessing a system's AI Bill of Materials (AIBOM) for weaknesses and vulnerabilities. The AIBOM extends the Software Bill of Materials (SBOM) concept which has been successfully used to identify supply chain vulnerabilities in commercial and open-source software. Specifically, the AIBOM extends the conventional SBOM to track the provenance, composition, preparation, manipulation, and consumption of data sources by an AI/ML system. This information is typically discarded during the creation and deployment of AI/ML models. However, it is necessary to perform critical T&E functions such as acceptance, operational readiness, performance and end-of-life testing of AI/ML systems. With Model-Inspector, the Army can create and maintain AIBOMs for commercial, open-source, and Government-built AI/ML systems that can be then used to automatically detect data drift, data leakage risks, and susceptibility to extreme, noisy, or adversarial inputs in the system throughout its operational lifetime. Model-Inspector directly addresses the need for scalable and robust T&E of AI/ML systems through a holistic approach to automated data and model supply chain management.Model-Inspector will provide the Army with a holistic and robust supply chain security management framework that is on par with the sophistication of the Army's existing hardware supply chain systems. Specifically, Model-Inspector will enable two critical vulnerability management scenarios. First, Model-Inspector can proactively warn systems developers when they train and/or deploy new AI/ML systems if the hardware, software, or dataset components they are using have known vulnerabilities and provide recommendations for remediation such as changing hardware settings, updating software library versions, or excising a dataset from the training regime. Second, the AIBOMs produced during training and deployment will allow for AI/ML system maintainers to periodically audit operational systems for newly discovered vulnerabilities identified after the model is deployed.