Nautical Evaluation of Mammal Observations (NEMO)

To protect marine mammals from ship strikes and sonar exposure, teams of human watchstanders maintain a tedious, costly, and daylight-only visual lookout. However, manned search is impossible on unmanned vessels like DARPAs ACTUV. Automated video search can provide affordable surface detections without human fatigue, and automated acoustic search can provide detections below the surface, but both approaches have limitations: IR only works in cold environments and hydrophones provide unreliable localizations. We propose Nautical Evaluation of Mammal Observations (NEMO), which combines EO, IR, and hydrophone sensors to enable long-duration surface and subsurface detection and classification (acoustic) coupled with high fidelity surface detection, classification, and localization in environments from cold or dark (IR) to warm or bright (EO). NEMOs Vision Module uses a coarse-to-fine approach combining state-of-the-art saliency analysis, appearance based detection, and deep learning classification in EO and IR to produce reliable, real-time marine mammal detection in all lighting and temperature environments. These visual search results are probabilistically fused with a best-of-breed commercial acoustic processing solution to produce detections with associated confidences in real-world coordinates. These detections can be used by an external autonomy system to select actions that best balance the conservation of at-risk marine mammals with mission objectives.