Point Cloud to 3D Textured Mesh Generation for Expeditionary Building Documentation and Preservation

As LIDAR (light detection and ranging) sensors become cheaper and more accessible than ever before, 3D reality capture has now become commonplace as evidenced by Apple’s iPhone Pro and Vision Pro devices. LIDAR scans produce a 3D point cloud which is a collection of millions of points in 3D space representing a physical real-world object. LIDAR is particularly useful for rapidly documenting buildings and spaces. The challenge, however, with 3D point cloud data visualization is that the resulting models are typically very heavy (difficult to share and view on the web), can have missing data due to occlusion, and be incompatible with most commercial building modeling software. Thus, a 3D textured mesh model is generally preferred for urban data visualization of buildings and structures due to its light weight and photorealistic surfaces.   The methods for obtaining 3D building mesh models can be divided into two categories: manual modeling and algorithmic generation. Although manual modeling provides the user with more control over details of the 3D model, this method is both time-consuming and labor-intensive, which is impractical for large scale urban documentation. Stitch3D has developed a prototype deep learning model that algorithmically calculates edges and faces from LIDAR point cloud data to rapidly create accurate 3D textured meshes. Specifically, our model consists of three steps: 1) 3D point cloud registration; 2) 3D model quality evaluation; and 3) 3D mesh generation using a 3D convolutional neural network (3D MeshCNN) with 3D self-prior. This SBIR Phase One proposal will explore the feasibility of training sparsely populated colorized point clouds using deep learning techniques to achieve a point cloud-to-textured mesh conversion suitable for large scale Air Force building documentation. If successful, research resulting from this SBIR Phase One effort will allow civil engineers and airfield managers to rapidly create digital twins of the built environment, whether for post-damage analysis, health and safety inspections, or existence and completeness (E&C) surveys.