Tracking the Little Things in Big Urban Scenes

Detecting and tracking dismounts in wide-area aerial video is a challenging task because of low resolution, parallax, occlusions from buildings and other structures, nuisance motion, and low frame rate on some sensors. We propose a multi-target tracking system that will robustly track dismounts and vehicles too in these scenes. Our approach addresses the challenges by using multi-frame analysis to accurately detect the small, low-contrast movers, and using a track linking framework to preserve identity across occlusions. The multi-target tracking system is designed to track individuals in and through groups, using a discriminative formulation which allows the accurate localization of a target while actively avoiding other targets and the background. Our approach easily incorporates domain knowledge and context such as road networks, scene categorizations (road, building, etc), and 3-d reconstructions. Moreover, our approach can model, learn, and utilize the dynamic patterns of the targets. For example, we can learn the expected paths taken by objects, and learn relationships between when and where objects are occluded, and when and where they reappear. This enables our approach, over time, to adapt itself to the scene. The result of Phase I will be an assessment of the difficulties of detecting and tracking dismounts in wide-area urban aerial video.