Contextual Observations for Natural Speech CharacterIzation On Unmanned Ships (CONSCIOUS)

Due to the necessity of operating alongside humans piloted vessels in open oceans and busy littorals, Unmanned Surface Vehicles (USVs) need a means to effectively communicate with such vessels when existing methods of safe navigation, including Automatic Identification System (AIS) and COLREGS, fail to fully address non-standard maneuvering scenarios. To achieve this, TDI Technologies, Inc (TDI) proposes Contextual Observations for Natural Speech CharacterIzation On Unmanned Ships (CONSCIOUS) to give unmanned vessels the ability to intelligently understand and interact with human piloted ships over VHF bridge-to-bridge radio. While many maritime inter-vessel interactions can be safely maneuvered using COLREGS standards, a large set of these interactions, including multi-vessel interactions, is not fully prescribed. When faced with these gaps in COLREGS specifications, mariners often turn to VHF bridge-to-bridge communications to negotiate safe passage. This is a challenge for USVs as they have difficulty understanding context and meaning in unstructured data, including spoken word. In order to successfully operate for long durations with minimal or no human intervention, it is critical to close this technology gap, and give USVs the ability to convey maneuvering intent to and understand the intent of human piloted vessels using Natural Language Processing (NLP).  The CONSCIOUS NLP engine is tailored to accurately identify the intent of a human speaker by (i) training its linguistic models with a maritime focused collection of texts (e.g. boating forums, boating handbooks), known as a corpus, and (ii) developing a Long Short-Term Memory (LSTM) classifier that captures conversational context by combining the critical information from previous transmissions with the most recent transmission. After a transmission, if warranted, CONSCIOUS generates a response to convey the USV’s intentions and converts it to spoken word using Text To Speech (TTS). After each conversation, CONSCIOUS computes a qualitative metric, using ship movement data (synthetic in Phase I and captured live from AIS in Phase II) that describes how well it understood human intent and how well it conveyed the USV’s intent, which allows the CONSCIOUS team to see the effects of software modifications and focus efforts on poor performing TDI’s existing work on the Navy’s Prototype USV (PUSV), make it well suited to understand the software interactions and interfaces on the Navy’s USV efforts. With this knowledge, the team will develop CONSCIOUS using Unmanned Maritime Autonomy Architecture (UMAA) standards and Data Distribution Service (DDS) for message handling as prescribed by current and future Navy USV efforts, allowing for a streamlined transition to the Fleet.