Spatio-Temporal Deep Learning for On-the-Move Threat Detection using Full Motion EO/IR Sensors

PhysicsAI proposes to develop a spatio-temporal video processing capability for exploiting full-motion EO/IR sensors for on-the-move real-time detection of in-road and road-side threats to support route clearance applications by combining several deep learning approaches in a way that has not been done before: (1) End-to-end deep learning – for mapping EO/IR sensor inputs directly into threat detection outputs resulting in more accurate, faster, and more robust models to provide real-time detection in full-motion video at 30 frames per second and higher, (2) Spatio-temporal deep learning – for providing a fully-integrated spatial and temporal machine perception of robust target features that evolve over time in full-motion EO/IR sensor data, including threat signatures that occur at different spatial and temporal scales, (3) Multi-branch DNNs – for fusing data from multiple video inputs in real-time in a way that is flexible and applicable to wide variety of sensors; and for producing multiple types of outputs such as threat classifications and detection of locations.