3D Reconstruction, Visualization, and Modeling of Buildings from Multiple Image Sources

Extended reality (XR) has emerged as a crucial tool supporting JSTO-CBD’s mission to defend warfighters against chemical/biological threats. Extended reality, including augmented, mixed, and virtual reality, coupled with accurate transport and dispersion (T&D) modeling of chemical/biological hazards, can provide critical situational awareness, enhanced live training, and immersive virtual training. To generate XR environments, JSTO CBD requires 3-D models of real-world terrain, buildings, devices, and weapons integrated from multiple data modalities acquired from a variety of sensor platforms. To meet this need, we will develop a tool that can rapidly, automatically, and accurately generate 3-D models fused from these multimodal sources and export the data in the necessary formats for XR applications and T&D modeling. Our tool fuses 3-D geometries from multimodal sources with robust surface generation at varying degrees of fidelity. It labels building features, such as doors and windows, that are important for accurate T&D modeling. It also infers interior building structure from exterior data sources.