Real time speech enhancement/noise reduction for the hearing impaired

PROJECT SUMMARY/ABSTRACT The current commercialization effort is aimed at solving the number one auditory complaint of people with hearing loss. Whereas modern hearing devices (hearing aids and cochlear implants) generally provide good speech understanding in quiet environments, users typically struggle to understand speech when background noise is present. Accordingly, effective noise reduction (one that allows good speech intelligibility) is the number-one requested feature by users of hearing devices. A long-sought solution involves a single-microphone algorithm, one in which the speech and noise are picked up by the same single microphone, capable of removing noise and improving speech understanding. The single-microphone solution is desirable because it is not subject to the constraints of multi-microphone systems (e.g., beamforming). The current AI revolution finally allowed such an algorithm, after 60 years of failed efforts. The initial demonstration of AI-based noise reduction was produced by the current scientific advisor. It allowed large improvements in speech understanding, but it was constrained to only the conditions it was trained on. The decade of scientific investigation that followed allowed the barriers standing between the initial demonstration and a real-world capable system to be overcome. The current development is focused on translating these scientific advances into a commercial product capable of removing background noise and restoring speech understanding. We will implement state-of-the art neural networks to perform noise reduction on the most common consumer portable device, the Apple iPhone. This platform has vast computational and battery power relative to ear-worn technology and is therefore capable of supporting relatively large and powerful networks. The result will be a software application capable of either (i) operating independently using earbuds or (ii) integrating with existing hearing technology (e.g., Apple Made for iPhone Hearing Aids). Neural networks will be broadly trained so that they can operate effectively on any voice and in any noisy environment. They will be implemented in a manner that allows real-time operation at a predetermined overall latency. Guided by these latencies, the parameters of the neural network will be manipulated to produce the most highly and broadly effective implementations. These implementations will be assessed using commonly employed objective metrics to establish their ability to remove background noise and improve predicted speech intelligibility and sound quality. Using test speech and noises not employed during network training, the real- time system will be evaluated using extended short-time objective intelligibility (ESTOI) and perceptual evaluation of speech quality (PESQ). Preliminary studies indicate that the system is capable of large improvements in predicted speech understanding, along with high speech quality, in real-time, and for essentially any listener in any noisy environment. The current development effort therefore has the potential to improve health and quality of life for millions of Americans by bringing to market a solution to arguably our most significant and one of our most intractable problems.