Multimicrophone Adaptive Array Augmented with Visual Cueing

DESCRIPTION (provided by applicant): Advanced Medical Electronics Corporation (AME) proposes to develop a smartphone-based adaptive audiovisual array that enables hearing aid users to converse with multiple speakers in reverberant environments with significant speech babble noise where their hearing aids do not function well. The array will consist of a smartphone, a smartphone accessory, and a smartphone application. The smartphone accessory will be a dongle that will house a multimicrophone adaptive array. The dongle will also contain a lower power radio by which it can transmit audio signals to compatible hearing aids. The smartphone software application will implement an adaptive audiovisual beamforming algorithm. It will use the smartphone's built-in camera to acquire images. Since the array leverages a handheld consumer electronic device, it will be portable and low cost. We will evaluate the prototype by conducting human tests in Phase I. PUBLIC HEALTH RELEVANCE: The primary complaint of hearing-aid users is their difficulty understanding speech in noise. Only 51 percent of hearing aid users report they are satisfied with their hearing aids in noise. A discrete device that augments hearing aids in difficult reverberant environments would allow hearing aid users to converse with multiple people in almost any environment including restaurants, automobiles, etc.