Hear What I Want: an Acoustically Smart Personalized Common Room

AbstractNoisy rooms with multiple active sound sources create problems for hearing-impaired listeners. Unwanted masking sounds reduce the quality and intelligibility of speech that listeners want to hear, especially listeners with hearing deficits. We propose a novel assistive listening system called HWIW (“Hear What I Want”) that “scrubs” (i.e., removes) noise and other unwanted audio components from complex real-world environments containing multiple acoustic sources. HWIW has been designed for integration into NIH’s Open Speech Platform initiative for hearing aids and other personal audio devices. HWIW will leverage STAR Corp’s Multiple Algorithm Source Separation (MASS) application framework of “pluggable” acoustic separation signal processing modules. MASS is compatible with the Open Speech Platform and available on GitHub.HWIW is a room-centric system that delivers listener-specific audio to users through their smartphones. HWIW employs multiple microphones distributed around a room and connected to a room-specific dedicated server. An initial HWIW setup procedure is used to name permanently positioned “noisemakers” in the room such as speakers and appliances and characterize their acoustic radiation and reflection patterns. The HWIW Room Server processes audio signals from multiple HWIW mics to scrub the noisemaker-generated sounds from any microphone in the room a listener chooses to monitor. Multiple listeners are supported simultaneous- ly. Each listener uses a HWIW Listener App to specify which mic to monitor for sounds of interest and which of the known noisemakers to scrub. The HWIW Room Server computes an individualized scrubbed audio stream for each listener and transmits it wirelessly to their Listener App. The Listener App outputs this audio stream to the listener’s hearing aid, personal audio device, or earbuds as a standard line level or Bluetooth audio signal.HWIW is room-centric, sensor image-based, latency-optimized, and listener-aware. Important system components are embedded in the acoustic space itself, rather than in the user’s ear (the hearing aid). HWIW calculates the acoustic image of masking sounds in sensor response mixtures so that images of unwanted sounds can be removed. It computes the latency of its signal processing and balances the quality benefits of longer-latency scrubbing against the perceptual advantages of faster response times. HWIW employs listener- specific acuity profiles, information about the sound-isolating properties of each listener’s hearing aid or ear piece, and the listener-specified masking sounds to determine whether which maskers are audible given the listener’s acuity; and thus what the optimal noise scrubbing strategy is for that listener.In Phase I, we will implement three HWIW MASS scrubbing modules, and a prototype of the Listener App. We will objectively measure the ability of the scrubbing modules to scrub noise from microphone responses, calibrate those measurements against perceived residual noise, and evaluate Listener App useability.The HWIW system will help hearing impaired listeners hear what they want more clearly in noisy rooms.