Multi-int Multi-look 3D Features Image Fusion with Machine Learning, Novateur Research Solutions LLC

Under this SBIR Phase II project, we propose a Transformer-based deep-learning framework for detecting targets from multi-INT airborne sensors. Building upon Novateur’s Phase I work, we enhance the generalized Transformer-based framework to fuse multi-modal sensor data in the feature space for improved target detection in complex scenarios. The proposed system takes advantage of self-attention in Transformers to capture long-term dependencies in multi-modal sensor data and estimates the relevance of the computed deep features from heterogenous sensors. The framework also exploits multiple views of the target from streaming sensors in an online fashion to keep improving its detection and classification accuracy over time. We will also develop methods for hard example mining and augmentation policies to optimize the performance for detecting occluded targets. This proposed effort builds upon Phase I achievements and integrates advanced technologies in feature extraction and feature fusion with deep-learning, The Phase II effort will focus on developing an end-to-end detection pipeline and a prototype software application to demonstrate the capabilities different sensors modalities on customer-provided and publicly available aerial data in representative scenarios. The proposed effort will build upon the experience and expertise of Novateur team in the areas of deep neural networks, sensor fusion, data exploitation, and development of real-time systems for robotic applications.