Scene Text Extraction and Translation for Handheld and Mobile Devices

We propose novel solution to develop and prototype a full automatic scene text detection and character and graphic recognition system for road sign translation which will run on a commercially off the shelf Personal digital Assistants (PDA). We propose to develop a prototype of a road sign/document translation system that can be run in a commercial of the shelf PDA. At the end of Phase-II effort, Army soldier will be able to use Polar Rains prototype software for road/informative sign translation and translate some of text section in document using their digital camera attached PDAs. They will point the PDA to signboard or document and press a single button to start transition. The system will capture the image of a foreign text and on board system automatically, extract the text from the image, recognize them, and translate to desired language, so they will able to see the foreign sign in their language in a less than 10 seconds. Recent advances in mobile device technology and high resolution digital camera availability for PDAs.This allows us to extend the capabilities of the proposed system by adding (a) fast low-level image processing techniques to enhance to image better text detection and character segmentation (b) feature extraction modules where the text identified, and skew and orientation distortion are corrected (c) a multi-level dynamic shape histogram based method for latin and non-latin character recognition (d) context aware machine translation module translates the recognized text in to desired language. (e) support other functionality, such as not only the road sign translation but also other informative text and graphical road signs and text in document translation (e.g., menu, advertisement, text on piece of a paper) (f) process single frames up to 1200x1024 resolution (g) increasing accuracy of the recognition and handle smaller size characters with new image super-resolution based magnification (h) live video-processing to use redundant information to obtain higher resolution and larger image to increase the recognition performance.