Non-Parametric, Closed-Loop Testing of Autonomy in Unmanned Aircraft Systems

The proposed Phase I program aims to develop new methods to support safety testing for integration of Unmanned Aircraft Systems into the National Airspace (NAS) with a particular focus on testing the collision avoidance (CA) algorithms of a UAS Sense-and-Avoid (SAA) system. The two primary issues addressed by this research are: (i) the risk that incorrect/mismatched modeling assumptions will skew simulation analyses and (ii) the fundamental difficulty of verifying the performance of autonomous systems that dynamically react to the environment. In particular, this research program would develop novel methods for conducting non-parametric, closed-loop simulation testing of collision avoidance algorithms. The technology generates a campaign of simulation experiments that automatically adapt to the algorithms in question. The purpose of this innovation is to expose potential vulnerabilities in UAS autonomy that are generated through the interaction of autonomous UAS algorithms with other agents such as an intruding aircraft operating under ``right of way rules". This work augments both the probabilistic open-loop testing methods, where agents do not react, and closed-loop testing where agent behavior is fixed a priori.