SBIR Phase I: American Sign Language (ASL) Science Technology Engineering and Mathematics (STEM) Thesaurus.

This SBIR Phase I project proposes to produce online software that enables deaf users to obtain English equivalents for American Sign Language (ASL) science, technology, engineering, and mathematics (STEM) signs. Many deaf and hard of hearing people who rely on ASL for communication do not have good facility with English, which is both spoken (and so aurally inaccessible) and written, resulting in poor reading achievement and subject content deprivation. This population has been particularly underserved in STEM literacy and under-represented in STEM careers. There are three primary reasons: First, nearly all scientific information currently available is English audio and text-based and without interpretation into sign language. Second, few educators investigate and use research that points to instructional practices that yield best results for deaf students. Third, few materials interpret STEM content into ASL and none from ASL to English. This proposed assistive technology will take a novel approach to assisting ASL users in understanding and appropriately using STEM terminology by creating the first-ever ASL STEM Thesaurus. Via menus, users will be able to describe signs using four descriptors (i.e., hand shape, palm orientation, location, and motion) and find corresponding English STEM terms, ASL and English definitions, and concept pictures. The broader/commercial impact of this assistive technology is its intrinsic value to a set of online assistive technology tools currently being developed whose ultimate goal is to improve literacy and educational achievement of deaf and hard of hearing students, particularly in science technology, engineering, and mathematics (STEM) fields. Having all of these tools on a central website makes instruction on STEM topics more accessible, efficient, and of better quality. Under a previous National Science Foundation grant, one of these tools, English-to-American Sign Language (ASL) translation software with a database of over 30,000 words, phrases, idioms, and symbols was developed. Because ASL is an unwritten language, translation output is represented as graphics and video. Further research was undertaken to recognize ASL using camera and sensor-enabled glove technologies for reverse translation (i.e., ASL-to-English). This project will use a sign language descriptor system in Phase I and camera/glove technologies in Phase II to enable deaf users to access STEM vocabulary from their own linguistic perspective. In this way, they will be able to better understand STEM terminology, navigate STEM content, and produce more cogent written discussions on STEM topics.