TeleSculptor:Semantic 3D Scene Modeling From High Zoom Reconnaissance Video

3D model generation from long-range ISR video is an important part of Special Forces operational planning. The current workflow is manual and time consuming. Existing automated algorithms for 3D reconstruction from high- and variable-zoom FMV are fraught with failure due to both practical issues and algorithm limitations that result in poor quality models. Kitware proposes TeleSculptor: Semantic 3D Scene Modeling from High Zoom Reconnaissance Video. TeleSculptor builds on Kitwares established software tools (Burn-Out, MAP-Tk, Super3D) that provide per-frame metadata burn-in detection, erroneous KLV handling, camera calibration for high-zoom FMV, dense surface estimation, and depth fusion. An end-to-end prototype of TeleScluptor will be delivered to AFSOC based on these tools for early feedback. TeleSculptor will be algorithmically improved by incorporating semantic segmentation into 3D reconstruction. It will segment video frames into semantic regions using the latest advances in deep learning. It will jointly estimate a 3D surface and a segmentation of that 3D surface into semantic categories. The semantic labels allow local geometric constraints that improve 3D model accuracy. Our proposal will enable fully automated reconstruction of high quality 3D models, saving countless man-hours and enable 3D exploitation of hours FMV that would otherwise sit idle in the archives.