AlphaUUV: A reinforcement learning approach to energy-optimized path planning for dynamic UUV missions

An Unmanned Underwater Vehicle (UUV) has to accomplish complex tasks while being subject to limited resource availability. Current UUV mission-planning tools manage the resources available before the beginning of the mission, and the UUV just “statically” executes predefined instructions. However, during a mission, UUVs can be subject to many unpredictable situations, such as obstacles, water current instabilities, and hardware failures requiring real-time decisions subject to resource constraints. The vehicle’s robustness to environmental variations is crucial for the outcome of the operations in an uncertain and dynamic environment. Many UUV applications abandon dynamic global path planning altogether because of the algorithmic complexity, and instead execute safe and reliable missions with perhaps some level of obstacle avoidance, at the expense of efficiency or potential information gain. Prior state-of-the-art systems have clustered around deterministic rules-based expert systems on one end, and relatively opaque deep learning-based systems on the other. Naturally, there exists an unprecedented opportunity to advance optimization and autonomy of UUV technology that exploits the best of both expert system and emerging deep learning breakthroughs. We propose developing, demonstrating, and commercializing AlphaUUV, a hybrid yet synergistic fusion of expert systems and deep learning algorithms to energy-optimized path planning for dynamic UUV missions. Building on the successful development and application of EpiSci’s Tactical AI framework for related autonomous system designs such as manned-unmanned teaming for pilots and drone wingmen, AlphaUUV technology innovation and tools development is envisioned to provide a highly effective, reliable, and explainable system, allowing for assured and bounded behavior and reliable command and control across a wide range of UUV platforms and mission objectives. The resulting technology can be used for both as a mission planning tool with comprehensive analytics and as a “self-diving” UUV controller for many UUV platforms. The latter is attributed to AlphaUUV’s modular design where it is capable of incorporating actual propulsion modules with no change to the overall AlphaUUV design. This is enabled by AlphaUUV’s modular, hierarchical, and hybrid machine learning structure rooted on Tactical AI.  AlphaUUV’s multi-objective path-planning capability is also capable of taking operational aspects into account such as command & control structures, communication types, components energy consumption, and battery/power-source characteristics. Upon successful development and demonstration of AlphaUUV technology and tools suite, we anticipate highly successful launch of AlphaUUV products for both UUV users (mission planners) as well as vendors enabled by its platform-agnostic, trusted, modular design features.