Learning-based Threat Classification and Localization Using Infrared Imaging and Ion Mobility Spectrometry

Concealed chemical threats represent a significant hazard to field operators. Even brief exposure to chemicals can cause lasting physical, physiological, and mental damage. Field operators require tools and techniques to more quickly, safely, and reliably locate and identify chemical threats. Chemical detectors typically excel at either threat classification or threat localization, but seldom both. Conventional sensor fusion techniques through probabilistic inference or fuzzy logic can be used, but these techniques do not exploit low-level features and data coupling between sensor modes that can often improve performance. Learning-based sensor fusion solutions using deep neural networks (DNNs) can surpass the best performance of each sensor individually, as demonstrated in recent work with multi-spectral pedestrian detection or audio-visual sound source localization1,2. In this Phase 1, we will implement an infrared imaging and ion-mobility spectrometer (IMS) multi-modal DNN to perform agent classification and localization and compare its performance to single mode DNNs. In Phase 2 we propose to improve and mature the DL architecture to multiple agents and deploy the solution to a government-built UAS to perform an autonomous CBRNE surveillance technical demonstration Concealed chemical threats represent a significant hazard to field operators. Even brief exposure to chemicals can cause lasting physical, physiological, and mental damage. Field operators require tools and techniques to more quickly, safely, and reliably locate and identify chemical threats. Chemical detectors typically excel at either threat classification or threat localization, but seldom both. Conventional sensor fusion techniques through probabilistic inference or fuzzy logic can be used, but these techniques do not exploit low-level features and data coupling between sensor modes that can often improve performance. Learning-based sensor fusion solutions using deep neural networks (DNNs) can surpass the best performance of each sensor individually, as demonstrated in recent work with multi-spectral pedestrian detection or audio-visual sound source localization1,2. In this Phase 1, we will implement an infrared imaging and ion-mobility spectrometer (IMS) multi-modal DNN to perform agent classification and localization and compare its performance to single mode DNNs. In Phase 2 we propose to improve and mature the DL architecture to multiple agents and deploy the solution to a government-built UAS to perform an autonomous CBRNE surveillance technical demonstration.