Fast Multi-Domain Battle Simulator (FMDS) for AI/ML

In our Phase I work, we created AfsimEnv, a reinforcement learning (RL) environment for the Advanced Framework for Simulation, Integration, and Modeling (AFSIM). We also showed that AFSIM is a suitable simulation environment for training a policy via RL in both speed and modeling capability. For Phase II, our goal is to improve upon AfsimEnv and the training scenario we first used in Phase I, as well as to develop new interfaces for both training an agent and utilizing the agent once it is trained. For training, we will develop a graphical user interface (GUI) called the Advanced Graphical Environment for Neural Network Training using Reinforcement Learning (AGENNT-RL). This GUI will allow users to easily select their RL training parameters and hyperparameters without having to modify Python code by hand or run the training from the command line. Once the agent is trained, it can be used in Warlock-AR, our new extended reality (XR) interface to AFSIM. Warlock is the existing GUI tool in AFSIM for operator-in-the-loop wargaming. Warlock-AR will be loosely based on Warlock, but designed for augmented reality (AR) devices, such as headsets or hologram tables. It will allow the user to view the agent acting in the environment and will include interactive commands for human-in/on-the-loop control. With these two interfaces in place, we will not only be able to provide a trained agent that can be judged by experts but it will allow users to create their own agents and utilize them for mission planning or other purposes.We will also demonstrate the use of AfsimEnv with government-owned, Stellar Science-developed Galaxy and the accompanying Dakota Toolkit from Sandia National Laboratories. With these tools, we will be able to optimize the RL hyperparameters, resulting in a much better trained agent. Galaxy also makes it easy to move from a regular workstation to a high-performance computing (HPC) environment. It is already available on the DoD Supercomputing Resource Center (DSRC) computers, and the GUI interface allows a user to set up a simulation and deploy it without writing any scripts. The combination with Galaxy will be both powerful and necessary to ensure a well-trained agent.