Feature Adaptation via Multispectral Extraction (FAME)

Tracking ground targets using full motion video (FMV) turrets is critically important to reconnaissance and targeting operations. A reliable automated system to track designated targets would free the sensor operator to perform secondary tasks, such as communicating intelligence to commanders. Current systems have not demonstrated sufficient reliability to gain the users"trust, working well under some conditions, but breaking under others. To help improve the reliability of FMV trackers, we propose an approach called Feature Adaptation via Multispectral Extraction (FAME), which adapts the feature sets used by the tracker based on operating conditions (time of day, sensor depression angle, etc.) An off-line learning phase is used to model tracking reliability improvements from varying feature combinations across different operating conditions, and the model is applied at run-time to select the best features to use. FAME also adaptively selects among multiple spectral bands based on operating conditions. The system will be demonstrated using video data from commonly-available sensors, such as infrared and the color channels from an electro-optical sensor, but the approach is extensible to an arbitrary number of image channels. FAME"s tracking algorithm uses a particle filter to model target uncertainty, incorporate geospatial knowledge, and fuse measurements from multiple sensors/spectral bands.