Offline Learning and Counter Artifical Intelligence for Autonmous Aircraft Combat Operations

Reinforcement learning (RL) consistently produces controllers that exceed human performance on complicated tasks in control and strategy. Despite this promise, their widespread use is limited by a few important problems. First, they are incredibly compute intensive. Popular RL algorithms do not allow the reuse of data during their learning. This problem results in inflexible controllers because controllers cannot be easily modified for a new task, and this constraint limits the number of controllers that can be deployed because each controller takes immense resources to create. Last, this inflexibility results in suboptimal controllers because controllers cannot easily learn from different sources of data, which includes examples from expert human operators. The second issue is that most controllers learned by RL are susceptible to counter AI attacks, which can force a controller to fail catastrophically. While this issue is not important for applications where environments are reliable and fair, DOD applications cannot operate under these assumptions. Toyon Research Corporation will develop new training methodologies that enable data reuse and the ability of RL algorithms to learn from related tasks. We will also research Counter AI attacks, and will develop methods to make controllers learned with RL more robust to these attacks.