Verification and Validation Software for Autonomous UAS Certification

VISIMO, a HUBZone small business, and University of Cincinnati will advance the verification and validation of Unmanned Aircraft Systems (UAS) by developing a testing environment (TE) that supports certification through the evaluation of safety and stability of artificial intelligence (AI) models used for autonomous flight. Utilizing a 3D simulation, the TE will test complex sensor fusion and ML decision-making routines. This system allows for testing of non-deterministic, ldquo;black-boxrdquo; models by evaluating their performance in a realistic setting integrated with flight software and hardware. By carrying out failure modes in the TE, the certification tests can expand to include cascading failures in emergency scenarios, simulating rapid changes in operating conditions such as incoming storms, sensor failures such as obscured camera views, and hardware failures such as a bad Inertial Measurement Unit sensor (IMU). The high-level architecture of our solution contains a test management system. This system provides a way to design test suites, evaluate test results, and decide on further tests to pin down the complex, subtle failures that can occur in multi-sensor AI systems. The flight simulators that currently allow failure injection do not contain this useful post-test analysis. This test management system will also run parallel tests faster than real time, greatly surpassing the speed of current single-instance user-run tests. The purpose of the proposed Phase I work is to advance state-of-the-art to create a TE capable of certifying that complex nondeterministic algorithms used in autonomous UAS flight are safe for integration into the wider NAS environment; and (in Phase II) to subsequently develop tools to validate algorithms for prognostics, vehicle health, sensor fusion, decision-making, and cooperative planning using the Phase I TE. Phase I work will result in a technology demonstration and final report detailing the TE proof-of-concept.