Shielded Deep Reinforcement Learning Approach for Small Satellite Swarm Intelligence, Surveillance, and Reconnaissance

Recent advances in satellite hardware have encouraged the proliferation of small satellites for a variety of mission applications; for example, minisatellites (10-500 kg), microsatellites (10-100 kg), and nanosatellites (1-10 kg) have become cost effective options for earth observation, space domain awareness, and communications missions. This widespread proliferation of constellations of small spacecraft drives a need for autonomy at swarm scale; it is not feasible to manually operate the large numbers of spacecraft involved in these constellations, and onboard actions must often be taken to respond on a much shorter timescale than is possible with a human in command.The Space Force is seeking methods to use and orchestrate orbital swarms to observe orbital and/or terrestrial objects through one or more sensor modalities. From the request for proposal (RFP): “awardee(s) will develop methods to use and orchestrate orbital swarms to observe orbital and/or terrestrial objects through one or more sensor modalities. Areas of interest in proposed solutions include, but are not limited to: synergistic sensing effects from swarm-based sensors; unique swarm-enabled capabilities; orchestration schemes for command and control of individual swarms and/or multiple swarms working on concert; orbital swarm formation management; swarm space-based awareness and proximity-sensing capabilities; swarm orbital deployment and initialization schemes; low-observable swarms; and swarm collaboration schemes with other on-orbit assets.” Our technical solution will address orbital swarm orchestration, synergistic sensing effects from swarm-based sensors, orchestration schemes for command and control of a single swarm, and swarm space-based awareness capabilities.            Trusted Space proposes to extend our distributed Shielded Deep Reinforcement Learning solution to balance: (1) tracking known objects, (2) search for lost and unknown objects, and (3) managing onboard resources in swarms of small spacecraft. Our solution will leverage information theoretic reward signals to incentivize agents to contribute to the global search and tracking objective. We will train our reinforcement learning model in the medium fidelity Basilisk simulation environment, and we will use the Blue Canyon Technologies (BCT) simulator to demonstrate that the performance of our model generalizes to a high fidelity spacecraft simulation. Our program objectives are to apply SDRL to a co-orbital or terrestrial search and tracking focused Design Reference Mission (DRM), extend the solution to an orbital swarm application, and demonstrate suitability for flight using the high fidelity Blue Canyon Technologies simulation environment.