SBIR Phase I: Ushering in a New Paradigm of Real-Time Tactile Graphics for Visually Impaired Students

This SBIR Phase I project aims to create a new form of tactile (touch) graphics for blind and visually impaired (BVI) students in Science, Technology, Engineering, and Math (STEM) education. To illustrate, consider learning in a traditional mathematics classroom without being able to see the graphics presented on the board. This is the challenge that BVI students face on a daily basis in STEM classrooms, and the current technologies available require intensive manual preparation, cannot be presented in real-time causing a delay in student learning, and are expensive. This project addresses these challenges by developing software that translates visual images into images that can be felt (through vibrations) and heard (through sound) on commercially available touchscreens. This work, rooted in understanding how people see through touch toward bettering the STEM learning experience, directly relates to NSF's mission. This project will push U.S. classrooms into new heights of learning and inclusion that will create opportunities for students of all learning styles to access, participate, and succeed in STEM classrooms and professions. This will have an economic impact through the creation of a competent, diverse U.S. STEM workforce and will position the U.S. as a clear leader in inclusive, accessible educational experiences. The innovation in this work is in conveying visual graphics via vibrations and sounds on touchscreens, such that an end user could interpret information from the image without needing to see it. Toward meeting this grand challenge, this project addresses a fundamental barrier to touchscreen-based tactile graphics identifying and following lines. Lines or combinations of lines form the basis for most graphic representations. Because touchscreens are flat, however, there are no physical edges or depth changes that can be used to represent lines, making line following on these surfaces (and subsequently interpretation of graphical information) incredibly challenging. Upon completion of this Phase I project, this work will have (1) demonstrated the feasibility of the algorithms and nonvisual representations needed to promote more efficient line following via vibrations and sounds on touchscreens and (2) demonstrated pilot software in three classroom settings with BVI students and their teachers to understand how the software could be integrated into traditional educational settings. This project will contribute to the movement of propelling touchscreens beyond visual input/output devices and will shed new light on how individuals perceive, interpret, and learn through vibrations, while creating a paradigm shift in what is considered today as tactile graphics.