Remmie.ai: a deep learning diagnostic assistance engine for ear-nose-throat diseases

PROJECT SUMMARY Otitis media (OM) is experienced by five out of six children before their third birthday, and 30-40% suffer recurring infections, leading to 16 million annual episodes in the US. Ear infections are the primary reason for antibiotic prescription for children under 6 years, are the second most common cause of hearing loss, and can lead to lifelong sequelae. Diagnosis depends upon in-person clinic visits and visual examination by care providers, at great inconvenience to patients and caregivers and at significant cost to the healthcare system, estimated at $4 billion per year. Although the majority of OM cases resolve within a week and symptoms may be managed by over-the-counter medications,10-20% do not, requiring additional antibiotic treatment or, in extreme cases, tympanostomy tube insertion to provide ventilation to the middle ear and aid in fluid drainage. Another compounding factor is limited access to otolaryngologists for accurate diagnosis and infection management. The expansion of telehealth has the potential to address this need with rapid, convenient, and affordable, but to date, there are no platforms to support and facilitate effective virtual visits for OM diagnosis. The first Specific Aim of this Phase I proposal involves building a comprehensive database of several thousand images of eardrums from patients with or without acute OM, with associated clinical diagnostic labels to, in Specific Aim 2, train a novel custom machine learning algorithm, Remmie.ai. A convolutional neural network will be developed to classify images of eardrums paired with text description of symptoms. Image classification will be improved through data augmentation, and the custom Remmie.ai architecture built through transfer learning of a publicly available training model. Unblinded labels will be compared to the algorithm readout as blinded testing data are loaded into Remmie.ai to ensure convergence of accuracy and validation for classification of acute OM versus normal eardrums. In Specific Aim 3, the Remmie.at platform, coupled with a handheld “portable otoscope” for imaging patients’ eardrums and a user-friendly mobile device application, will be tested by end-user physicians to derive feedback on the usability of the device and software. The outcome will be a novel tool for both patients and caregivers to monitor otolaryngic diseases, specifically acute OM, based on patient-provided images and symptoms, and diagnosis, aided by the proprietary Remmie.ai algorithm.