Multi-Domain C2 RL Training Environment: MUD CRANE

Boston Fusion Corp. (BFC) and MAK Technologies (MAK) propose MUD CRANE: Multi-Domain C2 RL Training Environment. The need for simulating multi-domain operations (MDO) is pressing, with the first Defense priority in the upcoming 2022 National Defense Strategy highlighting the “…growing multi-domain thread posed by the PRC [People’s Republic of China]”. MDO presents both opportunity and challenge for Joint Forces as it provides a plethora of options to commanders to execute simultaneous and sequential operations. The tempo and complexity of command and control (C2) decision-making increase in MDO scenarios, with forces seeking windows of superiority for temporary dominance over an adversary. MUD CRANE will allow war planners to leverage advances in the field of artificial intelligence (AI), particularly within deep reinforcement learning (DRL), to support their decision-making. These algorithmic approaches have proven adept in numerous strategic settings, beating human champions at the strategic games of Go and StarCraft 2. MUD CRANE will support applying these techniques within the military decision-making process (MDMP) to enable faster, more robust course of action (CoA) planning support in MDO. Furthermore, MUD CRANE will support Army researchers seeking to develop relevant, cutting-edge DRL agents in support of this mission. Effective DRL approaches require simulation environments which rest on the three pillars of very high speed execution, multi-domain richness/fidelity, and specialized AI/ML interfaces. An environment must execute fast enough to generate massive amounts of training data for DRL techniques in an operationally relevant window of time, must include enough realism for DRL agents to learn meaningful strategies transferrable to the battlespace, and must include flexible interfaces enabling DRL model experimentation, updates, and replacements. MUD CRANE will offer: (1) A high-speed, multi-domain, operationally relevant high-fidelity simulation environment which is broadly accepted within the DoD community with built-in scalability and open communication standards; (2) Flexible machine interfaces for training DRL agents, allowing for user-defined observation and action spaces and plug-ins for calculating custom reward functions. Additional interfaces will include tools for generating stochastic scenarios varying force laydowns and compositions, Red and Blue CoAs, as well as environmental factors such as weather and time of day; (3) A framework for human interaction with the DRL training process, creating a teaming environment for CoA generation; (4) Plug-ins for quantifying custom metrics, including CoA complexity measurements; (5) Well-documented exemplar DRL agents for experimental demonstrations; and (6) A researcher-centric framework allowing each entity to be controlled by a DRL agent, default (scripted) behaviors, or user-defined behavior trees.