Soteria Phase II

Recent advancements in Machine Learning and Computer Vision (ML/CV) have demonstrated the ability to both enhance existing Naval systems and enable entirely new capabilities that would have been impossible with traditional methods. While this period of rapid advancement is exciting, the opaqueness and brittleness of ML methods, specifically deep-neural networks (DNNs), make it difficult to reconcile with traditional system and software airworthiness verification procedures and has prevented their widespread adoption within the Navy and DoD despite their excellent performance. To expedite adoption and equip our soldiers with the best performing tools available, there is a need to establish methods to guide system developers trained in traditional airworthiness through the process of selecting, implementing, testing, and verifying the performance and safety of ML/CV methods onto existing and emerging platforms. System designers must also be equipped with reliable methods of predicting and mitigating failures of ML methods prior to their deployment and effective real-time checks to monitor component and system level performance while systems are in operation. To address these concerns, La Jolla Logic (LJL) and Sierra Nevada Corporation (SNC) propose the Soteria Toolbox; a collection of ML capabilities and complementary methods for verifying ML/CV performance and mitigating risks and uncertainties. Team LJL combines LJL’s experience developing a wide range of ML models and user interfaces and their capabilities for developing complex software systems with SNC’s vast expertise in a wide range of safety and mission critical applications. SNC is an industry leader in both landing systems and aerial docking as well as multi-sensor fusion, ISR, and air platform integration. As an example, SNC provided the guidance, navigation, and control (GNC) for the first aerial refueling exercise and, more recently, implemented the GNC for DARPA Gremlins providing aerial recovery of UAS in flight. This effort will combine SNC’s experience with LJL’s expertise in AI/ML to develop a collection of ML/CV models that will enhance the performance of fielded systems with supporting tools to ensure they are operationally safe and adoptable by traditional system developers. As the primary focus of this effort, the operational safety and verification components will focus on methods that both verify the individual ML/CV components and aid in developing system level effects and mitigations. This will leverage both recent advancements in ML verification and traditional statistical and outlier rejection techniques. LJL’s expertise will be leveraged to curate selected models and capabilities to achieve the optimal performance with SNC’s experience developing complex safety critical systems. The team will ensure that the developed Soteria Toolbox and its curated models are accessible to engineers who work on these systems and that the verification and mitigations processes.