Reading Signs in Context to Support Perception-Based Autonomous Navigation

Success of autonomous vehicles and driver assistance systems depend on their ability to interpret and operate in unknown environments with little a-priori information. To be successful, systems developed to operate in these environments must detect,recognize and reason about various textual and graphical content from signs, labels, and plaques. Our approach builds upon previous R&D to provide a real-time architecture for detecting, tracking, and recognizing signs from dynamic video imagery. Ourapproach detects signs by combining visual cues consisting of color, shape, and text. Signs are tracked to improve detection accuracy and provide resiliency to occlusions and environmental effects. The orientation of the sign is determined and the imageryrectified for recognition processing. Text and graphics characters are extracted using a combination of optical character recognition and syntactic/semantic parsing. Our Phase II program will address the areas of the system design that were not fullydeveloped during Phase I, including the use of feedback between processing components, fusion of information in order to reason about the location, size, movement, and appearance of signs, and active detection, tracking, and recognition of signs, where thesensors? parameters are actively controlled in order to overcome difficult viewing scenarios.