4D Dynamic Scene Modeling for Situational Awareness

ABSTRACT: We propose to develop a systematic framework for constructing 3D scene knowledge from EO/IR video sequences to provide accurate 3D change detections, informative shadow/occlusion masks for target tracking, and geo-location and compression indicators. We construct 3D scene models at a range of fidelities to support robust modeling over a broad range of imaging geometries and viewpoint diversities. 3D models are constructed efficiently by leveraging scalable distributed processors as well as a sparse volumetric reasoning algorithm that separates parallax from ground motion using just a few image frames. The 3D modeling processing is extendable to integrate data from other 3D sources and the 3D change detection processing is extendable to integrate change indications form multiple sensing modalities. The 3D scene modeling and analysis capabilities provides a practical and rich source of high-fidelity spatial constraints that is fully up-to-date and immediately exploitable for downstream scene understanding applications that require accurate target detection, tracking, and/or geolocation in urban and high relief environments. BENEFIT: The automated development of 3D models from aerial video data will enable high performance ISR applications in complex urban and high terrain relief environments. 3D scene models provide strong spatial constraints for focusing target detection on areas of interest, such as the ground plane, confidently identifying parallax effects, and predicting occlusion and shadowing effects. Keys to the success of a practical tool to accomplish these functions is the integration of 3D data from multiple viewpoints and modalities, ability to process video data from arbitrary collection geometries, and efficient processing of computationally complex 3D modeling algorithms. The 3D modeling and analysis tool will generate of actionable 3D-derived data products such as 3D change detection, 3D geolocation, shadow projection, occlusion projection, and compression quality indicators for use in a wide range of defense, security and geographic information system applications.