Gaze Steered Microphone Array for Understanding Speech in Noisy Areas

Project Summary Abstract Advanced Medical ElectronicsAMEproposes the development of a novel portable wireless hearing aid accessory that utilizes head pose and eye tracking of the hearing aid user to steer a microphone array in the direction of interestA common problem for hearing aid users is the difficulty of participating in conversations in a noisy environment such as at a workplace or restaurantWhile hearing aids can improve speech intelligibility in certain types of noiseit is difficult to distinguish between multiple talkers where some might benoiseand others are notTwo of the best solutions for improving hearing aid performance in these scenarios includemoving a separate companion microphone closer to the source oraiming a beamforming array microphonewhich is more sensitive to sound in a particular directiontowards the desired sound sourceWhile companion microphones have been shown to be effectivethey have drawbacks that have limited their useCompanion microphones require an active involvement of the speakers to use microphonesThere are many situations where this is impractical or socially awkwardThere are also microphone beamforming array solutions on the marketbut there are challenges with steering or pointing the array to the source of interest as well as making the array unobtrusiveThis Proposal presents an innovative solution for using a beamforming array microphone that solves the issues of being cumbersome and slow to pointespecially in a fast moving conversation of many peopleThe front facing camerathe side with the screenon a smartphone will be utilized to detect the head pose and eye direction of the hearing aid user s focus of attentionThe microphone arrayattached to the back of a smartphonewill be electronically steered with signal processing to collect sound from the direction the user is lookingThe processed audio will then be transmitted directly to the user s hearing aid via a digital wireless linkThe user will simply hold the smartphone in a natural manner for someone using a smartphone and look at the person in a group they want to listen toIn Phase IIAME will build a production prototype system and test it with hearing impaired individuals