Deep Reinforcement Enabled Warfighter Assistant

The Missile Defense System (MDS) is a complicated system with enormous state spaces, real-time operations, limited time to consider the Courses of Action available, and dire consequences for suboptimal defenses. Operating this complicated system requires the operators to make constant tradeoffs to improve the Probability of Engagement Success, minimize Interceptor Wastage, and enable the fight to go longer. Allocation of the finite number of interceptors during raids can truly be the difference in successfully defending millions of lives. A computational decision aid can revolutionize the missile defense fight and enable our warfighters to optimize interceptor allocation decisions. Moreover, these types of difficult tradeoffs exist in all areas of military and commercial sectors. Computational decision aids have the potential to assist humans in parsing through the immense web of interactions in our environment, increasing the value our decisions return. Current state-of-the-art decision-making algorithms utilize Deep Reinforcement Learning (DRL). DRL has made massive breakthroughs, surpassing grandmaster-level play in games and defeating a fighter pilot in a simplified simulation. It is not enough, however, to merely have a DRL warfighter assistant that makes brilliant decisions. This technology must communicate its reasoning behind decisions and integrate into the timeline in which humans execute their decisions. This is not currently achieved by high-performing DRL. Current work towards explainable Reinforcement Learning (XRL) has only delivered handicapped algorithms on easier tasks. In complex tasks, the decision logic itself is high dimensional and requires dimensionality reduction above merely transparency. nou Systems is pioneering interpretability in the DRL space through applying explainability methods in a novel way to graph based DRL algorithms. Graph neural networks (GNNs) provide dimensionality reduction and white-box interpretability into the model. Reward decomposition incorporates key mission metrics into the agent's thought process. We can communicate the tradeoffs made by the agent through value contrasts on the mission metrics. Communicating Shapley Values on top of reward decomposition provides mathematically sound rationale for the model's recommendations. Our approach allows the model to be freely trained by state-of-the-art DRL algorithms without complications caused from explainability methods. Furthermore, our graphical explanations are crafted to be rapidly understood to enable the warfighter to approve or alter decisions while the engagement is transpiring. Approved for Public Release | 24-MDA-11673 (8 Jan 24)