Deep Sea Vision AI: Deep Learning-Based Real-Time Marine Animal Detection for Entanglement Mitigation

TECHNICAL ABSTRACT: The objective of this project is to develop and implement a real-time detection and warning system to mitigate marine entanglement events in offshore aquaculture operations. In order to achieve this, we leverage the latest advances in computer vision, deep learning, and real-time object detection to develop a real-time marine life detection, classification and tracking pipeline: Deep Sea Vision AI (DSV-AI). We deploy our real-time detection pipeline on robust yet inexpensive off-the-shelf (OTS) hardware. The complete DSV-AI detection system is designed to be usable in multiple configurations, including from Unmanned Aerial Vehicles (UAVs), stationary, pole-mounted or marine bouy-mounted platforms. Phase I is primarily focused on developing an demonstrating the feasibility of the deep learning and computer vision-based software pipeline. This pipeline consists of several software components to: 1) acquire real-time video, 2) pre-process (de- noise/image enhancement) video frames, 3) detection and classification of marine animals of interest, 4) real-time tracking, position and velocity calculation, 5) activation of deterrents to discourage entanglement, and 6) provide real-time alerts and supporting data to stakeholders.SUMMARY OF ANTICIPATED RESULTS: The image annotation platform developed during Phase I be further extended to Phase II and utilized to annotate and classify thousands of images for all of the marine animals of interest. The hardware and software combination developed during Phase I may be adapted based on lessons learned during the performance profiling and testing stages. This will aid in informing Phase 2 work, particularly with regard to specific system components which may need to be modified based on results of the feasibility study and prototype developed during Phase I. The convolutional objects detection model utilized during Phase I may be supplanted by a faster or more accurate model during the time required to complete the Phase I effort and prior to or after commencement of Phase II.