SPECTACLES (SPECTrum Agnostic multi-Camera optimaL mEtrics fuSion)

Imaging systems comprised of electro-optic (EO) and multiple infrared bands are being deployed due to their ability to provide passive situational awareness that is robust to environmental conditions. Fusing multi-sensor multi-spectral image streams to provide an operator with a single unified image and improving downstream algorithms performance using the fused image is a challenging problem. SPECTACLES will provide a plug and play real-time multi-sensor multi-spectral image fusion system. Inputs are pre-processed to allow varying camera properties and asynchronous image streams to the system. The explicit optimization-based fusion module optimizes across all spectral bands to provide the highest quality image for a human operator and robustness to environmental conditions.  An offline optimizer adapts the fusion objective function to improve a specific downstream algorithm’s performance. SPECTACLES will be demonstrated on recorded datasets and benchmarked against accepted fusion metrics. SPECTACLES inputs and outputs will be compliant with relevant MISB, MISP, and NITFS standards to ease integration and transition to imaging systems of interest for the Navy.