Miner/Istis for Passive Object Detection (MIPOD)

As the types of threats faced by the U.S. Navy evolve, developing sensor capabilities to track and identify military entities becomes increasingly important as well as ever more complex. Since utilizing powerful active sensors sometimes risks sacrificing stealth, it is highly desirable to develop tracking and identification software that takes advantage of passive sensing capabilities in the electro-optical (EO), infrared (IR), and radio electromagnetic bands, as well as any available sonic passive sensing capabilities.ÝThe complexities of multi-observer, multi-spectral passive object detection present a unique opportunity for Stottler Henke Associates to leverage systems we have previously developed for multi-view image fusion and ìtrack-before-detectî object identification. For multi-view image fusion, we developed the MultI-view NEural Radiance (MINER) prototype for fusing pixel-level data between views. In Phase I, we demonstrated that this technique exceeds the performance of state-of-the-art multi-view fusion techniques on a simulated object detection task. For track-before-detect identification, we developed theÝIntelligent Surface Threat Identification System (ISTIS) to automatically analyze the data associated with surface tracks, hypothesize, draw inferences, and make ID-related recommendations. These data include the trackís location and/or velocity reported over time along with other ID-related reports such as IFF codes, visual ID reports, intelligence reports, communications, etc.ÝTogether, the ISTIS and MINER prototypes present the opportunity to develop Miner/Istis for Passive Object Detection (MIPOD), a prototype that integrates both systems in an innovative pipeline to provide multi-spectral, multi-observer passive object detection with state-of-the-art performance. The proposed MIPOD architecture will integrate the innovations from both MINER and ISTIS as well as incorporate novel modifications to adapt each prototype for the passive object detection problem.ÝBased on naval, aerial, and overhead sensing inputs, we extract radio, sonic, EO, and IR data relevant to the identification of a specific target or set of targets (as well as associated metadata, such as the sensor pose for EO and IR imagery). MINER will fuse EO and IR imagery collected by different observers, resulting in a set of probabilities over possible identities for each target as well as the estimated 2D location of the target within each sensor frame. This information, along with any detected radio and sonic signals and associated metadata, will be passed to ISTIS, which will produce localizations and suggestions for identification (ID) for each target. In an operational prototype, MINER will fuse information between views, while ISTIS will be integrated with the ACS console GUI to provide operators with maximal explainability and interpretability, enabling an informed final decision regarding the target ID.