Structure from Motion

The Real-Time Structure From Motion (RTSFM) system combines state-of-the-art techniques and novel methods to compute 3D models of battlefields in real time from calibrated image sequences. Unlike most computational stereo approaches, the work is tailored for long sequences of calibrated imagery under near-planar small-baseline motion – sequences typically generated from loitering unmanned aerial vehicles. The stereo correspondence approach is based on simulated annealing, a stochastic iterative optimization technique, and depth estimates are also influenced recursively by previous estimates. An abstract 3D surface model based on a hybrid Voronoi-BSPT (binary space partition tree) representation is incrementally generated and refined, enabling low-bandwidth incremental transmission off the aerial platform. RTSFM is a combined hardware/software system with some critical system components implemented in field-programmable gate arrays. This approach facilitates novel methods for optimizing image frame processing order, for seeding stereo correspondence calculations, and for recursive camera parameter estimate refinement. Together these reduce the computational requirements of the system such that accurate real-time 3D modeling is feasible. RTSFM directly increases the warfighter’s ability to execute combat operations by improved situational awareness. It also enables improved approaches for automatic target recognition, operations against aboveground and underground structures, battlefield damage assessment, and many other tasks.