SBIR Phase I: iGlasses: An Appliance for Improving Speech Understanding in Face-to-Face Communication and Classroom Situations

This Small Business Innovation Research (SBIR) project will advance the state of the art in human machine interaction, speech, machine learning and assistive technologies. The innovation in the proposed research is to develop and test the technology required to design an embellished eyeglass, which will perform continuous real-time acoustic analysis of the interlocutor's speech and transform several continuous acoustic features of the user's speech into continuous visual features displayed on the eyeglasses. Pilot research has demonstrated that it is possible to recognize robust characteristics of isolated auditory words and to transform them into visible features in real time. The proposed research extends this research to sentences along with tests of different feature detectors and automatic recognition models. The proposed activity will impact society by providing a research and theoretical foundation for a system that would be available to all individuals at a very low cost. It does not require literate users because no written information is presented as would be the case in a captioning system; it is age-independent in that it might be used by toddlers, adolescents, and throughout the life span; it is functional for all languages because it is language independent given that all languages share the same phonetic features with highly similar corresponding acoustic characteristics; it would provide significant help for people with hearing aids and cochlear implants; and it would be beneficial for many individuals with language challenges and even for children learning to read.