Multimodal Sensor Fusion and Exploitation for Proliferation Detection

The Department of Energy’s National Nuclear Security Administration Office of Defense Nuclear Nonproliferation is tasked with detection and identification of new and emerging sources of illicit production and transportation of nuclear-related materials and technology. A variety of sensor modalities providing data at various spatial, spectral, and temporal resolutions exist to aid the detection of targets of interest such as nuclear fuel production sites. Exploiting literal and non-literal data at multiple spatial, spectral, and temporal resolutions requires expertise in many technical aspects of imagery analysis, and is challenging and time-consuming. Many sensors have specialized algorithms unique to their system, requiring specialized analysis and visualization tools. Additionally, some non-literal systems like Synthetic Aperture Radar and Hyperspectral Imaging require special training and a unique set of algorithms for interpretation. To address the challenges presented by these federated systems. Automated fusion algorithms that could manage different sensor modalities collected at different times and from different geometries would greatly enhance analysts’ current ability to extract actionable intelligence from this variety of sensor data. Under this effort, a rigorous approach to multimodal data fusion will be developed as the scientific basis for combining data streams from multiple imaging sensors to address this problem. The algorithms and framework developed here will provide the basis for combining and exploiting a variety of sensor modalities such as synthetic aperture radar, hyper-spectral and panchromatic imagery, video imagery, lidar pointclouds, or other sensor types that may become available. The fused data picture that the framework will produce will improve and accelerate the current analysis chain by improving automated detection and classification algorithms relevant to the Office of Proliferation Detection as well as providing a more informative and intuitive visualization and data interface for Proliferation Detection analysts. In Phase-I the data fusion algorithms will be developed and implemented in a prototype software environment that will be demonstrated against relevant government provided data. Based on this success, a Phase-II effort would be used to mature and refine the algorithms and implement them into an end-to-end fusion framework. The work done under this effort will benefit proliferation detection analysts by making them more effective and efficient at detecting and monitoring nuclear fuel production facilities. The same algorithms would benefit a number of commercial and government programs that use commercial or satellite imagery.