Scene Text Extraction and Translation for Handheld and Mobile Devices

We propose novel solution to implement a full automatic text detection and identification system for road sign translation which will run on a commercially off the shelf Personal digital Assistants (PDA). The system consists of a PDA, a digital cameraattached to a PDA, and a storage device which can be attached compact flash II ports of the PDA. The fast and memory efficient system consists 4 modules for road sign translation: (a) low-level image processing techniques to enhance to image better textdetection and extraction, segmentation (b) feature extraction modules where the text identified, and skew and orientation distortion are corrected (c) a shape histogram based method for latin and non-latin character recognition (d) a limiteddictionary-context based machin translation module translates the recognized text in to desired language. The translated text is augmented on to same location where the original foreign text was, so user will sees the road sign/text in their language.As the system is full automatic text detection interaction between user and program will be minimal and simple. User need only point the device to road sign and press a button. In the Phase-I effort, we will implement a prototype system to evaluate thefeasibility of the methods. System will support a language pair in the phase-I effort, and it will include multiple language support as a Phase-II effort. First commercial applications of our proposed image processing technology will be in the areas of navigation aids for people in foreign country. People will get the software in their mobile devices and use them whenever they need a short translation, suchas, time-table in metro station at Tokyo, a Chinese menu in Beijing, shopping in market place at Dubai. The other initial private sector applications we will focus on will address the needs of vision-impaired people to function comfortably andindependently. Our applications will help them use the public transportation system by announcing the number and destination of an arriving bus, navigational signs in the metro system, identify street signs, and name of stores in shopping malls. Populationof senior citizens has been increasing steadily in developed countries, and along with that the number of vision impaired people who can benefit from such a product. In recent study shows that there are 5.5 Million visually impaired people in US only whomay not read the sign and text even they are very close the signboard.