MetaMosaic: Metadata Guided Geospatial Mosaics from FMV

The construction of geo-registered mosaic images from full motion video (FMV) can provide significant value to imagery analysts. When FMV provides high resolution video it also lacks in spatial context. A mosaic of video frames can help provide that context while maintaining high resolution. However, producing an accurate mosaic from operational FMV can be extremely challenging. Operators frequently pan or zoom the sensor rapidly and change between sensing modalities. FMV also frequently has metadata overlays burned into the pixels on screen, moving foreground objects, compression, and transmission artifacts. Kitware proposes MetaMosaic to improve mosaic generation by leveraging the metadata in the stream, either key-length-value (KLV) encoded or recognized from onscreen text. We not only read metadata values from the screen but also remove those metadata overlays so they do not produce artifacts in the mosaic. Our approach combines sub-pixel accurate image feature tracking with global location constraints from metadata to optimize a high quality mosaic even under challenging conditions. We propose a baseline 2D image registration as well as a full 3D solution for more complex scenes and camera motion. To remove barriers, we propose our FMV mosaic generation tool and all source code with unlimited rights to the government.