Audio-Tactile Interactive Computing with the Livescribe Pulse Pen

This SBIR project is designed to develop a system for delivering audio-tactile computer applications that are accessible to high school and college students who are print-disabled. The applications will include: a scientific calculator, a periodic table of elements, subway maps and a game. In each application, the user will touch the tip of a special pen to raised-line and textured (tactile) images printed on plastic film. As the user does, the audio system for the pen will play spoken descriptions of features he/she touches. Tests will be conducted with human subjects at two sites that will expose students with visual impairments to each application to study their performances and actions while using the system. It is hypothesized that vibrations and other haptic sensations delivered into the user’s hand as the pen’s tip is guided through a variety of grooves, channels, dimples and punched holes will bring an interactive richness and control without adding to tactile graphical clutter. By overlaying tactile images with additional information that is relatively transparent to fingertip touching, we will be able to display more information without increasing cognitive load, thereby improving comprehension of potentially complex figures. Anticipated Results and Potential Commercial Applications Technology developed over the course of this Phase II project will lead to the introduction of a suite of applications for audio-tactile interactive computing. It will also lay the foundation for the company and its partners to generate additional software titles for use with the Livescribe Pulse Pen. Considering the low-cost and capabilities of this device, and given a growing emphasis on universal design and assistive technology in the classroom, sustained commercial success with these products is likely and this research and development project has an excellent chance of spawning a whole new category of educational materials in the United States and Europe in the near term. In addition, experimental findings generated may potentially provide important new insights about the role of haptic feedback via a rigid stylus in human-computer interactions in general.