Reinforcement Learning For Coordination And Control of Swarming Satellites

Inspired by frequent observation of repetitive learned swarm behavior exhibited in nature, this novel program will develop and demonstrate new capabilities in decentralized control of large heterogeneous vehicle swarms limited in communication, sensors, and actuators, with direct application to communication-less coordination. These goals are accomplished through the adaptation and use of Reinforcement Learning solutions to the optimal control problem. Reinforcement Learning approaches define a value function, which represents the total reward for possible actions at a given state, deriving a decentralized formulation for each agent in a Multi-Agent System. The proposal implements the policy gradient method for Reinforcement Learning applied to swarming spacecraft control. Three major tasks are proposed for the development of swarming space vehicle coordination and control: Approximate Optimal Control for Large Swarms, Communication-Less Swarm Coordination Implementation, and Human-Swarm Interactions via Supervised Reinforcement Learning. Algorithm development in Phase I will extend to a Centralized Optimal Control Solution, Inverse Reinforcement Learning for the Local Decentralized Problem, Model Free Learning, "Expert Solution" Conversions to the Local Modified Local Interaction, Inverse Learning for Behavior Determination and Classification, Hyman Designed Dynamic Reward Functions, and Keep Out Zone Models. Follow-on efforts will are proposed for full implementation of the Reinforcement Learning swarm technology for real-time integrated system use and mission integration, including laboratory demonstrations of small robotic units, and the development of flight-qualified software and hardware packages for full integrated technology demonstrations.