Reinforcement Learning for Flight Control

The control of high-performance aircraft presents a number of challenges including multiple nonlinear subsystems, rapidly changing dynamics, unstable and/or non- minimum-phase modes, and performance criteria that change as a function of mission or pilot preference. As a result, flight control laws have required numerous iterations of an expensive design cycle whereby a controller is developed, tuned, and validated in a lengthy sequence of flight tests and design or parameter changes. Most adaptive methods that deal with these complexities cannot "remember" what they have learned about the aircraft dynamics and are extremely difficult to validate in terms of stability and robustness. In this proposal, Barron Associates, Inc. describes an unsupervised reinforcement learning (RL) simulation-based control design methodology that can, over time, learn an approximately optimal control strategy for complex systems with minimal analyst involvement. A controller for both inner-loop and outer-loop pitch-axis control of a fighter aircraft will be developed using the RL algorithm; the evaluation functions required by RL will be approximated using polynomial neural networks (PNNs), and a method for incorporating stability and robustness metrics directly into the RL design process will be investigated.