3-D Reconstruction, Visualization, and Modeling of Buildings From Multiple Image Sources

Extended reality (XR) and transport and dispersion (T&D) modeling are crucial tools for supporting the JSTO-CBDÆs mission to effectively defend warfighters against chemical/biological threats. XR, including augmented, mixed, and virtual reality, can provide critical situational awareness, enhanced live training, and immersive virtual training. Coupling digital XR environments with T&D modeling also enables fully realized hazard consequences, while T&D modeling alone enables advanced threat analysis. To support these tools, the JSTO-CBD requires 3-D models of buildings, terrain, devices, and weapons generated from opportunistically available data of various modalities (e.g., electro-optical/infrared/X-ray images, video, LIDAR, CAD, blueprints, topography, vector maps) collected from various platforms (e.g., soldier?work, drone, satellite, mobile phone). However, there is no tool that can rapidly, automatically, and accurately generate 3-D models fused from multimodal sources of various quality and export the data in the necessary formats for XR applications. Moreover, there is no tool that can interpret these generated 3?D models (i.e., identify windows, doors, and airflow elements) for automatically generating T&D simulation tool inputs. To meet this need, Creare is developing the Semantic Understanding and Mapping of Environments (SUMOE) framework, a modular and extensible software product that iteratively fuses available data to generate high?fidelity 3-D models.