SpatialGen - Generate 3D Spatial Environments For Analysis & Planning From Video

From paintings to sculptures, from blueprints to CAD models, humans have always sought to visualize the world in 3D.ááSpatialGen is proposing a novel use of AI to generate full-scale 3D digital twins for combat operations, reconnaissance, and disaster response. A map would no-longer be a 2D display from a satellite, but a 3D map like from a video game.ááFor mission planning, a leader would gain a fully immersive 3D view of real-world locations. They would take into account rocks, hills, floods, cars, and more when planning an operation. They would be able to see the operation from their hypothetical position in the real world.áThis software has 3 main components.áCapturing of videoProcessing of video into 3D environmentsAnalysis and Decision Making Application of the ResultáFor capturing the video, a simple consumer-quality smartphone can be used. Drone footage, bodycam footage, or novel spying equipment would also work. The resulting video is then sent to the server for processing. A simple prototype iPhone application shows this integrated with SpatialGen today.áThe processing of the video is where the technical nature becomes complex. Frames of the video are extracted and analyzed, then a custom state-of-the-art AI neural network is overfitted to the data set to generate a highly-detailed point cloud. This point cloud can be rendered into a smaller USDZ model for easy transmission and internal sharing. Today, these point clouds are often considered gaussian splats. Due to our startupĺs background in 3D video processing and creation, we know that additional fidelity can be extracted from consumer devices today than what is currently available. Depth data and stereoscopic methods can be applied to increase the fidelity of the result.áThe last component is a new application where key decision makers can assess the environment in 3D. This can be conducted on a Windows, Linux, or Mac machine. Notes can be added in 3D space, areas can be highlighted, operations planned, or the user could even put on a VR headset to step into the exact world. The end software application leverages WebGL and other technologies necessary to render 3D environments.