SBIR Phase I: Drone Control in Turbulence via Reinforcement Learning

The broader impact of this Small Business Innovation Research (SBIR) Phase I project is to improve drone control. Current controllers are effective in specific environments but perform poorly in environments and flight conditions such as turbulence, thereby narrowing the scope in which drones can be used. This project will advance a plug-and-play solution in which users can focus on higher-level tasks specific to their use case, like obstacle avoidance and route planning. This new controller has broad applications in both commercial and military settings: it enables stable flight across a wide array of environments, expands the flight envelope in turbulent conditions, and allows for longer missions due to increased control efficiency. This Small Business Innovation Research (SBIR) Phase I project addresses the problem of drone control in turbulence through the development of a reinforcement learning-based flight controller. The project will enlarge the design envelope for quadcopters as well as provide a system and environment for testing reinforcement learning algorithms that can be applied to other control problems. Contemporary systems rely on Proportional Integrative Derivative (PID) controllers as an essential part of stable flight. These PID controllers rely on holistically tuned, static functions to convert maneuvering commands into voltage changes across drone motors to meet the rotor’s targeted rotation speed. In lieu of statically defined PID equations, this novel controller uses a reinforcement learning algorithm, which is a subset of machine learning where an agent is trained to select actions that maximize a reward across an environment. This technique has led to greater-than-human performance across a variety of different control and game theory tasks, but little is known about how these techniques fare when replacing PID control systems. The primary advantage the development of a reinforcement learning controller would have over simpler PID controllers is the ability for the user to view drone control at a higher level of abstraction, thus mitigating the need to focus on the minutiae of flight control for complex missions. This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.