Registration of EO and LIDAR using an Interoperability Architecture for Net-Centric Environments (RELIANCE)

The “see first, understand first, act first, finish decisively” principle requires that military commanders and analysts be provided a complete, up-to-the-second view of the battlefield. Currently deployed full motion video (FMV) sensors provide crucial real-time situational awareness, but suffer from limited field of view and lack of geographical context. Augmenting current FMV displays with 3D light detecting and ranging (LIDAR) data, geographic information, and real-time intelligence would provide the user with a better understanding of video content in the context of the battlefield. Precise geo-registration is required to enable this type of display; however, existing solutions are limited to specific equipment configurations and have long processing timelines. To address these shortcomings, we propose a system for Registration of EO and LIDAR using an Interoperability Architecture for Net-Centric Environments (RELIANCE). RELIANCE features an interoperability layer to allow integration of various commercial and government off-the-shelf (COTS/GOTS) sensors and visualization software. Geo-registration is performed using a hybrid 2D/3D bundle adjustment technique to refine initial estimates from direct geo-referencing. To handle all types of environments and terrain, bundle adjustment uses two types of tie points selected from video frames: one from 2D-to-2D matching using reference imagery, and one from 3D-to-3D matching using LIDAR data. BENEFITS: This system could be used across all ISR platforms within DoD and law enforcement agencies where accurate geo-location and FMV sensors are used to track or observe targets. In addition to offering great benefit to force protection and perimeter surveillance applications, it could also be used for mapping applications, commercial robotic vision systems, agriculture and environmental studies, and to augment civilian surveying equipment.