Scene Estimation & Situational Awareness Mapping Engine (SESAME)

Unmanned Ground Vehicles (UGVs) must have self-localization capabilities, not just in abstract units like longitude & latitude, but with reference to real terrain. The ideal solution uses both onboard sensing (for real-time local scene information) andpreloaded digital maps (for a global perspective). We propose to develop a Scene Estimation & Situational Awareness Mapping Engine (SESAME) to accomplish this goal for UGVs such as Utah State University/TARDEC's T3. Our design will use mature commercialoff-the-shelf (COTS) stereo cameras and computer vision processors, enabling us to deliver a robust, fully-functional system at the end of Phase II that does not rely on exotic, high-cost hardware. In Phase I, we will specify hardware requirements (andoptionally evaluate available COTS stereo camera vision systems & acquire one), design the overall system architecture and all algorithms, and implement & evaluate software prototypes for key algorithms. Our mapping engine will generate localhigh-resolution digital elevation maps (DEMs) in real-time from stereo input and integrate with preloaded low-resolution DEMs. Our scene understanding algorithms will use color and shape to classify objects by material class (grass/foliage vs.rock/concrete, etc.) and category (tree vs. road, etc.). Situational awareness is then derived from correlating detected objects with known locations.SESAME has direct commercial potential to numerous DoD, DoE, and private industry UGV projects as asituational awareness module designed for low-cost COTS stereo cameras. The developed situational awareness algorithms also have excellent commercial potential as a spin-off product for the computer game industry: a software development kit that enablesgame developers to easily add sophisticated path planning capabilities to their games' built-in artificially intelligent opponent.