AlphaUSV: A Reinforcement Learning Approach for Safe and Precise USV Autonomous Navigation

Maritime areas such as harbors, channels, and straits require precise piloting of vehicles as they are typically congested with other unmanned surface vehicles (USV) and hazards (i.e. buoys, bridges). Current harbor piloting systems use a human to integrate numerous inputs including his or her sense of sight and hearing. To overcome the need for a human expert, a straightforward solution for autonomous navigation would be to set thresholds that are based on the input sensor data received to statically execute a predefined set of commands for each particular situation. However, while expert systems policies are characterized by rapid initial capability, their deterministic behavior provides vulnerabilities in specific and complex scenarios.  EpiSci proposes developing, demonstrating, and commercializing AlphaUSV, a hybrid yet synergistic fusion of expert systems and deep learning algorithms for safe and precise navigation of UUVs in congested maritime areas. AlphaUSV, with its expert system and deep learning integration as its core brain, provides a highly effective, reliable, and explainable system, allowing for assured and bounded behavior and reliable command and control, even with limited, degraded, or intermittent communications.  EpiSci is highly qualified to develop and demonstrate the proposed AlphaUSV system with extremely low development risks as evidenced by prior and ongoing successes on similar autonomous sensing and navigation technologies such as small unmanned aerial systems (1st Prize: NSWC Crane AISUM CRANE Challenge), unmanned aerial systems (Top 3 in the DARPA ACE AlphaDogfight program), and UMAA-compliant autonomous UUV technologies (AlphaUUV feasibility demonstration via digital twin). Consequently, we plan to leverage in-house, state-of-the-art computer vision and deep reinforcement learning (DRL) algorithms to detect objects using sensors such as cameras and lidar, estimate with precision their trajectories, and plan for a safe path with high confidence. The problem described in the AlphaUSV program is analogous to the AISUM CRANE Challenge but in an maritime domain.? We will leverage our winning solution in the CRANE challenge for this program.  Furthermore, EpiSci can leverage real-world experience from a UCSD collaboration with Naval Information Warfare Center (NIWC) Pacific through the Naval Research Enterprise Internship Program (NREIP) focusing the development on key aspects for the design of autonomous navigation for USV.