SBIR Phase II: Reconstructing Consistently Detailed City-Scale Environments From Incomplete 2D and 3D Data

The broader impact/commercial potential of this Small Business Innovation Research (SBIR) Phase II project is the ability to more completely understand the real world through a perfect virtual copy. This SBIR Phase II project makes it possible to create virtual reproductions of real cities that not only show how the world looks, but reproduce every object and every detail perfectly, allowing users to interact with it. These virtual copies make it possible to quickly and effectively train soldiers and first responders, train autonomous ground and flying vehicles, place new construction into a dense city, simulate the effects of catastrophic weather, and explore imaginary scenarios through games. Billions of dollars are already spent mapping and 3D modeling the real world for these and many other applications, but these virtual cities are created through painstaking manual methods that can take months to years, or lack necessary information about what is in the world. This SBIR Phase II project will make it possible to rapidly and automatically create detailed 3D copies of any area of the real world. This Small Business Innovation Research (SBIR) Phase II project will advance the state of the art in reconstructing highly detailed 3D models of the world for diverse commercial applications. This project introduces new methods for turning raw multimodal sensor data into semantic information describing the world and immersive, interactive-ready 3D models. It will remove the time, money, and manual effort necessary to create accurate 3D models of real world areas today, by using computer intelligence instead of human effort to parse sensor data like photographs and laser scans. The resulting 3D models and underlying semantic information describing the world will be used directly in game engines and simulation software, in analysis tools, in rendering software, and beyond. This research will build on the associated Phase I project, first improving the detail that can be identified and reproduced in virtual copies of the real world, then showing readiness for commercialization with paying customers. The result of the project will be a market-ready product for an initial market segment, capable of accurately reproducing real cities in 3D models that customers can readily use, as well as traction that demonstrates customer need for the product. This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.