SeeStar a mobile pda phone based assistance system for visually impaired users

DESCRIPTION (provided by applicant): The purpose of the proposed research is to continue development of the SeeStar system based on Phase 1 proof of concept results. Participant experiences with SeeStar under a variety of use conditions demonstrated the value of the system, and indicated where further development efforts will be required. SeeStar offers persons who are blind a pair of eyes via the use of a cell phone camera and data link to a remote assistant who can look at live images through that ce ll phone camera and then describe the person's surroundings, indicating the location of salient landmarks and pathways to specific destinations. Also, using GPS, the assistant can employ Google Maps to orient the person to their farther reaching surroundin gs. The experiences of Phase 1 participants, as well as remote operators, made it clear that continued SeeStar development should focus on static rather than dynamic use scenarios -- that is, not attempt to provide blow by blow directions as the person i s walking. Further, participants made it clear that in the absence of an available remote assistant, an automated means of detecting salient visual landmarks and specific destination points would be most useful. The participants were most enthusiastic when landmark and text recognition algorithms were demonstrated in this regard. The experience of the remote operators of SeeStar also highlighted specific issues to be addressed in Phase 2: (1) difficulties in communicating caused by video and two-way voice r eception latencies greater than 1 second caused by fluctuations in data channel capacity, (2) difficulty in orienting to a setting given the very restricted field of view of the camera lens, (3) low video image resolutions made it difficult to read signs a nd recognize objects easily, and (4) GPS coverage and errors. (5)Further, participants were split over whether they felt it was easier to hold the camera in their hands, or wear it around their neck, or perhaps even on their head. (6)Participants also want ed a means of verifying their intended routes (indoor and outdoor) and destinations, locating and recognizing pedestrian crossing buttons, bus stop signs, bus or train numbers, elevator buttons, exits, bathrooms, office numbers, and street signs. The devel opment the automated capabilities will include a significant amount of existing of state of the art but off the shelf, image based landmark recognition, text sign detection and recognition, and automatic panorama creation software. Such landmarks and text signs when referenced to a GIS database can provide a redundant method of positioning to GPS which may suffer from coverage and availability issues. The combination of object and text recognition will also enable location and identification of critical sal ient features in the absence of an assistant such locating and recognizing bus numbers, street signs, elevator buttons, or pedestrian crossing buttons. Finally, more extensive and statistically relevant focus group and end user tests will be conducted to i nsure we have adequately addressed issues of concern for this very heterogeneous population. PUBLIC HEALTH RELEVANCE: Using a from the consumer's perspective, SeeStar must level the playing field in a manner not yet in evidence for: In order to effectiv ely function within society, all people must have the ability to plan comprehensible routes, follow routes, and maintain orientation along the way. People must also be able to traverse any given location without being struck by moving traffic, confounded b y new construction sites, or stumbling upon a decaying sidewalk or roadway. People who have visual impairments face unique challenges in achieving independent mobility, as there is no easy way for them to identify potential hazards, or to immediately acces s information about their surroundings without an assistive technology device or assistance from a sighted person [T2RERC's Stakeholder Forum, 2003].