4D Dynamic Scene Modeling for Situational Awareness

ABSTRACT: We propose to develop a systematic framework for integrating the dynamic automatic generation of 3D scene models from aerial EO/IR video data with the extraction of moving targets in the scene. Our proposed system ingests raw aerial video and outputs a dynamic and compact 4D model combining the 3D scene background and the dynamic activity in the scene. This 4D model not only reflects the 3D background structure and the overlaid dynamically moving targets, but also represents the coupling between the movers and 3D scene at the appropriate level of detail. The coupling between the movers and scene includes both spatial constraints that impact the trajectories of the movers as well as the visibility constraints that explain limitations in the detection/tracking processing. The key enabler for this 4D modeling concept is the Volumetric Inference Engine. This engine, which dynamically and continually reconstructs the scene, is an enhanced version of the volumetric modeling approach developed by Pollard and Mundy. As part of its operation, the Volumetric Inference Engine also performs 3D motion detection which facilitates high-performance tracking by reliably detecting movers with very few false alarms, since parallax and glints are explicitly discounted. BENEFIT: The automated development of 4D models from persistent aerial video data will enable high performance ISR applications in complex urban and high terrain relief environments. The integration of 3D scene modeling with target detection/tracking processing will lead to improved exploitation results and the combined spatio-temporal data will provide analysts and downstream processing systems a compact and illuminating representation that elucidates how targets are moving through a 3D scene and how the 3D scene is constraining target trajectories and limiting or enabling sensor abilities to clearly see them. A practical tool for creating the 4D model that is part of the image analyst workstation will thus significantly enhance the analyst's productivity and quality of exploitation results.