MyDocSaid: Deep Learning for Patient-Centric Record, Summarization, and Analysis of Medical Conversations

Abstract Patients retain and recall less than 10% of medical care plans correctly after 5 to 7 days leading directly to poor comprehension, engagement, compliance, and poor health outcomes. This pervasive and detrimental dynamic is a significant contributor to large waste in the annual US healthcare spending now exceeding $4 trillion with a conservative estimate of 25% or $1 trillion annually. On the other hand, patients often find managing their healthcare overwhelming with complex/multiple diagnoses, specialists, and medications prescribed by different providers with a higher impact on older adults. Typically provider visits frequently increase with age. For a variety of reasons, aging patients generally experience an increasingly difficult time remembering and understanding the details of what was discussed during their office visits. Adult children of aging patients often assist and participate in their parent’s medical care but often can’t accompany them to provider visits. Unfortunately, existing technology falls short of meeting these patients’ needs. Almost all existing health recording tools are provider-centric and none are truly and completely patient-centric allowing patients to record provider interactions with natural language processing (NLP) translation, searchability along with full patient control, data mobility, education, and real-time artificial intelligence (AI) support. Potentia Analytics Inc. proposes the development of its patient-centric health recorder app, called MyDocSaid, to address these issues. MyDocSaid will be designed to provide patients with a secure, fast, searchable, and extremely user-friendly technology enhancing patient recall, remembering, comprehension, and engagement in their own health maintenance or restoration. This technology platform will also play a pivotal role in advancing the dynamic of “shared decision-making”. At-risk populations like senior citizens could benefit from the MyDocSaid application. Preliminary studies at Potentia Analytics show that the state-of-the-art automatic speech recognition (ASR) and NLP models will be able to solve this problem and provide transcription, summary, and keywords from medical conversations. To develop the back-end and AI components, Potentia Analytics will use transfer learning to fine-tune language models for medical conversations. Demonstrating feasibility in Phase I requires a working AI pipeline (including ASR/NLP models and knowledge graphs (KG)) that can pass usability tests in a production environment. The specific aims to accomplish this goal are: 1) Train and implement deep learning models for ASR & NLP tasks. 2) Train/implement KGs and deep learning models to recognize medical named entities and their relationships. 3) Integrate the data/AI pipeline. Medical terminologies will be detected using knowledge graphs (KG) trained on medical ontologies and will provide information about the context of the medical conversations. The fine-tuned AI models and knowledge graphs can be used by other researchers to further advance NLP in the medical context and ontologies. The front-end algorithms and the user interface will be designed for seamless use by elderly patients.