Passive Ranging in Low-Light Conditions using Fused EO/IR Multi-View Imaging

We propose a passive ranging technology that combines new low-light sensors with a novel multiview imaging to perform deep learning-based target positioning and tracking.Ā Our approach is based onĀexisting TRL6 software developed by Third Insight for for embedded devices and unmanned aerial systems (UAS).ĀĀThe software runs on lightweight edge devices, so can be deployed to ATAK, small CPU/GPU modules, IVAS, and UAS in support ofĀdismounted soldiers, in particular. The proposed approach is unique and designed to exceed US Army specifications:Ā The core multiview geometry algorithm accumulates a sequence of views-on-target using EO/IR full-motion video.Ā Targets are identified and classified as semantically relevant objects using deep neural networks (DNNs), which makes it easy to incorporate DNNs trained specifically forĀlow-light conditions and multispectral sensor fusion.Ā An integrated pipeline for tracking targets utilizes both traditional key-point signatures and DNNs.Ā Targets are tracked in 2D imagery and projected intoĀa virtual 3D environment to support ōscene understandingö and navigation, while building a database of target features for target re-acquisition during periods of occlusion or loss of observability.Ā A lightweight version of Simultaneous Localization and Mapping (SLAM) optimized for low-light sensors then performs small-region 3D mapping in parallel with our multiview algorithm.Ā Both techniques utilize similar key-point signatures and DNN-based features, allowing fusion of traditional SLAM withĀobject-based methods that areĀeffective when target recognition is available and/or useful for fire support planning. Our multiview algorithm has been tested on-boardĀUAVs to track and position targetsĀin a virtual 3D environment to support UAV navigation and fires planning.Ā These field exercises for AFSOC focused on small UAVs using EO cameras, which already show a range accuracy of better than +/- 20 meters at 300 meters for targets such as vehicles and personnel.Ā Phase I effort will extend our prior efforts and testing to include ground-based observations for individual soldiers that observe their target(s) from a small set of neighboring positions using new low-light imaging devices for passive nighttime operations.Ā We propose to additionally retrain our DNNs to obtain learned features that work with low-light datasets and recent capabilities for light enhancement.Ā Phase I will assess the impact of light quality on our updated multiviewĀalgorithmsĀfor range accuracies at ranges up to 1,000 meters with detailed analyses of performance for Phase I reports.