Chemical-Imaging Registration and Multi-modal Analysis

To address the Chemical and Biological Defense (CBD) agency’s need for a new deep learning (DL)-based threat detection solution that fuses imaging sensors and chemical/biological sensors to detect and locate concealed chemical threats, Intellisense Systems, Inc. (Intellisense) proposes to complete the development of the new Chemical-Imaging Registration and Multi-modal Analysis (CIGMA) solution. CIGMA will comprise advanced onboard sensor data analytics software running on a commercial off-the-shelf (COTS) embedded accelerated computing platform (such as NVIDIA Jetson Nano) that fuses data from chemical and imaging sensors in real time for accurate detection, localization, and identification of concealed chemical threats. CIGMA consists of multiple DL-based image analytics tools and algorithms to detect containers of potential concealed chemical threats. By spatially registering the containers and chemical sensor readings, CIGMA can pinpoint the source of a chemical threat. Thus, the CIGMA system not only improves threat detection confidence but also differentiates chemical threat types (such as chlorine, tear gas/pepper spray, and nitrogen oxides). It also increases location accuracy of the chemical threats (better than 5 m accuracy), significantly improving countermeasure capabilities. As a result, this system directly addresses the CBD requirements by offering improved threat detection confidence, identification, and location accuracy. In Phase I, Intellisense successfully demonstrated the feasibility of the CIGMA system by establishing the concept of operations (CONOPS), designing a DL-based system architecture, developing and prototyping key processing and fusion algorithms for the imaging sensor and chemical sensor, and demonstrating their improved threat detection, classification, and location capability in laboratory simulated environments. In Phase II, Intellisense will mature the CIGMA design for a field-portable computational platform that can accept and fuse multiple sensor inputs. The Phase II CIGMA prototype will meet the low-SWaP requirements for deploying the computation platform on small, unmanned vehicles (UxV). The Phase II prototype will be tested and evaluated on multiple threat vectors in both laboratory and operationally relevant environments with participation of government personnel to demonstrate improved threat detection and identification over single sensor DL architectures. At the end of Phase II, CIGMA will be ready for initial testing for potential transition and integration of selected imaging sensors and chemical sensors.