Electro-Optical and LIDAR Sensor Fusion for Self-Calibrating, Low-Cost UGV Perception

Sensor fusion allows a robot to overcome many of the drawbacks of individual sensors, but such techniques require precise calibration of all sensors. In support of the Navy's requirements to develop low-cost and robust sensors, TRACLabs and SwRI propose to design and demonstrate a combined EO/LIDAR perception system that facilitates improved autonomy for UGVs. We call this system LEOPARD (for Lidar and Electro-Optic Perception with Advanced Recalibration Design). Research and development of the LEOPARD system will focus on integration of the multi-camera/LIDAR hardware and on the software needed for automatic calibration, sensor fusion, and terrain analysis. Calibration is needed for both the intrinsic parameters and extrinsic sensor parameters. Methods exist in both academia and in commercial software for automatic calibration of multi-camera systems. Some of these use calibration targets, while some can calibrate using arbitrary environmental cues; however, none of these methods have been integrated into a robust fieldable system. In addition, calibration between a multi-camera vision system and 3D LIDAR sensors is less well developed. We expect to unify various calibration methods into a single optimization framework, such that a deployed LEOPARD system will automatically detect and correct calibration errors in the field, without intervention by military personnel.