Mobile Search for the Visually Impaired

DESCRIPTION (provided by applicant): IQ Engines' mobile visual search technology will enable the visually impaired to access real-time information about physical objects using their mobile phone camera. The mobile phone provides a visually-driven hyperlink between the physical and digital world: point the camera at an object and get information (for example product information or navigation information). The mobile phone camera is a powerful yet underutilized tool for the visually impaired. Our proposal has two specific aims. Working directly with the visually impaired community, we will build a prototype mobile visual search application that meets their accessibility and use requirements. Our second aim is to improve upon the state of the art for 3D object recognition. We will investigate a novel combination of sparse image representation, feature matching algorithm, and geometric verification in order to advance the performance of 3D object matching. While state-of-the-art image intelligence is robust enough to enable rapid and accurate image search of flat feature-rich objects, current computer vision pales in comparison to the abilities of biological vision systems to recognize 3-dimensional objects. Our underlying goal is to bring inspiration from recent advances in theoretical neuroscience and apply them to image and video search solutions. PUBLIC HEALTH RELEVANCE: Mobile visual search, using a cell phone camera to retrieve object information, enables a mobile phone camera to become an artificial 'eye' with object recognition intelligence. Implemented on a cell phone, a mobile visual search tool can be a low cost visual aid for the blind.