Automated Entity Classification in Video Using Soft Biometrics

Military and intelligence operations are gathering imagery from a variety of sources including ground-based sensors, aerial platforms, and satellites at rates far exceeding our ability to analyze or even transmit and archive this data. This imagery comprises not only still images but increasingly full-motion video (FMV); high-definition (HD), wide-area imagery (WAMI), and other high-resolution formats; and multi- or hyperspectral data encompassing the visible and infrared spectra. While much of this data is ultimately not operationally useful, deployed assets lack the ability to perform in-situ analyses and identify the relevant elements. In addition, there exist vast databases of other auxiliary and contextual information, yet these data sources cannot be linked as it is impossible to query the image data directly in a semantically meaningful way. Therefore, there is an urgent need for automated tools and techniques that are able to process, triage, analyze, and enrich image and video data with more descriptive metadata. To bridge this technology gap, we will develop a suite of analytic capabilities for processing a wide variety of image and video data and producing metadata annotations describing regions of interest (e.g., persons, objects, contextual information).