Detecting and Classifying Marine Mammals in Infrared Video Using AI

We propose to develop a system that can automatically detect marine mammals day and night for extended periods of time using infrared video cameras and custom software. We plan to make use of commercial long wave infrared (LWIR) cameras. For vessel‐based data collections we will use a custom camera developed by Toyon that stabilizes imagery electronically using data from a low‐cost inertial navigation system (INS). Stabilized video from multiple imagers will be sent to a detector where human‐ designed algorithms will be used to find potential locations of marine mammals. Video will be clipped spatially and temporally around each detection and the resulting video clip will be sent to a deep neural network classifier that uses artificial intelligence (AI) techniques to classify the video clips. Further development of the AI module will be the primary focus of this effort and this will entail the collection and formatting of large volumes of data as well as the evaluation of multiple architecture models to determine the best for this application. A fully functioning real‐time system will be created and demonstrated to NOAA staff as well as potential Phase III partners