Learning-based Direct Dense Mapping

To address the Army’s need to compile information from multiple frames acquired on a moving unmanned aerial vehicle (UAV), Intellisense Systems, Inc. (ISI) proposes to develop a new Learning-based Direct Dense Mapping (LDD-Map) technology based on multiframe depth/distance and camera-pose prediction with unsupervised deep learning and fast point-cloud alignment. Specifically, an innovative convolutional neural network (CNN) architecture explicitly models the geometry of camera pose change and image transforms to infer three-dimensional (3D) scenes at the pixel level from sequential video frames. The 3D scenes acquired from multiple look angles and multiple imaging systems are fused with efficient optimization algorithms to form a consistent 3D ground map and generate feature vectors. As a result, the LDD-Map technology offers accurate registration of video frames to ground locations, consistent 3D feature vector generation, and simultaneous UAV localization and mapping, which directly address the Army requirements. In Phase I, ISI will demonstrate the feasibility of LDD-Map by conducting preliminary testing of its core algorithms on contractor data, and study the impact of various sensor and platform factors, such as frame rate, magnification, range, and look angle. In Phase II, ISI plans to implement LDD-Map using Government data and processing hardware suitable for UAV platforms.