Speech Perception Training Advanced Scoring and Feedback Methods

PROJECT SUMMARY Speech perception in noise is a pervasive frustrating problem for older adults with mild to moderate hearing loss Few of these adults qualify for or use hearing aids Auditory speech recognition training is considered a possible intervention but research has failed to deliver reliably effective training regimes to ameliorate speech recognition difficulties Training to improve visual speech perception may be more effective because older adults may experience no significant declines in their ability to combine auditory and visual speech information visual speech information delivers significant benefit for overcoming acoustic noise interference and the benefit increases with better lipreading ability Most individuals who have relied throughout their lives on auditory speech are not good lipreaders Consequently there is wide room for improvement in visual speech perception and audiovisual benefits under difficult listening conditions Additionally the few existing training systems available to older adults do not incorporate current knowledge about visual speech perception and the underlying mechanisms of perceptual learning This Phase I project will further develop and test a visual speech training system In Aim we will port our existing laboratory lipreading training system to the cloud and perform an initial informal evaluation of its operation and in Aim we will formally test the efficacy of the cloud based training with hearing impaired adults who have been screened and receive pre and post testing at George Washington University A training system that presents sentence stimuli and carries out open set recognition response analyses and feedback was developed for this project The core system components are phoneme to phoneme stimulus response sequence alignment and computational modeling of the perceptual dissimilarities of visual spoken phonemes and words The system can produce diverse types of feedback Three will be compared in Aim Consonant Word and whole Sentence feedback During training a video sentence is delivered and participants type their open set response The system analyzes the response Consonant and Word feedback give feedback on correct and perceptually near response words Sentence feedback is the conventional printed whole sentence feedback Aim evaluation will determine whether there are significant differences across feedback approaches for lipreading and for audiovisual speech recognition in background noise Individual differences in the benefits of each feedback type will be examined Focus groups in Aims and will be convened to give the project team feedback on areas such as ease of system use understanding of system instructions and feedback formats The project will lead straightforwardly to Phase II which will include activities such as developing a larger stimulus database including online audiovisual testing materials and obtaining normative data on system efficacy across age groups and hearing loss levels Project Narrative Auditory speech recognition training for hearing impaired HI older adults has not delivered adequate benefits for ameliorating the problem of speech recognition in noise Seeing the talker helps overcome the noise but benefits are smaller for individuals with poor lipreading ability A home computer based lipreading training system for improving audiovisual speech recognition in noise will be further developed and tested in older HI adults