A Specialized Automatic Speech Recognition and Conversational Platform to Enable Socially Assistive Robots for Persons with Mild-to-Moderate Alzheimer's Disease and Related Dementia

Abstractinseniors in the United States dies with dementiaof which Alzheimer s diseaseADis the most common formAD patients suffer from decreased ability to meaningfully communicate and interactwhich causes significant stress and burden for both professional caregivers and family membersSocially assistive robotsSARshave been designed to promote therapeutic interaction and communicationUnfortunatelyartificial intelligenceAIhas long been challenged by the speech of elderly personswho exhibit age related voice tremorshesitationsimprecise production of consonantsincreased variability of fundamental frequencyand other barriers that can be exacerbated by the neurological changes associated with ADfurther complicated by common environmental noises such as the ceiling fantelevisionetcBecause of the resulting poor real world speech and language understanding by available SAR technologiesscarce human caregivers are often required to guide AD patients through SAR interactionslimiting SARs to small deploymentsmostly as part of research studiesUnlike existing approaches relying purely on AIcare coachis developing a SAR like avatar that converses with elderly and AD patients through truly natural speechEach avatar is controlled by axteam of trained human staff who can cost effectively monitor and engageor more patients sequentiallysimultaneouslythrough the audio visual feeds from the patient s avatar deviceThe staff communicate with each patient by sending text commands which are converted into the avatar s voice through a speech synthesis engineThe staff contribute to the system their human abilities for speech and natural language processingNLPand for generating free form conversational responses to help patients build personal relationships with the avatarThe staff are guided by a software driven expert system embedded into their work interfacewhich is programmed with evidence based prompting and protocols to support healthy behaviors and self careThis SBIR Fast Track project will leverage the unique data generated by our humanin the loop platform to develop new ASR capabilitiesenabling fully automatic conversational protocols to engage and support AD patients without human interventionWe aim in Phase I to leverage our unique prior work dataset to train an automatic speech recognitionASRengine to enable the understanding of certain types of elderly and AD patient speech more successfully than any currently available engineWe aim in Phase II to incorporate this new engine along with an NLP module into our existing human in the loop avatar systemrecruiting a population of AD patients to further train and validate with during ayear human subjects study so that we can demonstrate full automation of a significant portion of our avatar conversations with mildto moderate level AD patientsThuswe will improve the commercial scalability of our avatarswhile validating our new ASR NLP engine as the most accurate platform for enabling the next generation of AD focused SARs Narrative Artificial intelligenceAIhas long been challenged by the speech of elderly personsand especially persons with dementiadue to age related voice tremorshesitationsimprecise production of consonantsincreased variability of fundamental frequencyand other barriersUnlike existing approaches to socially assistive robotsSARsrelying purely on limited AI for conversationcare coachhas been commercializing a SAR like avatar that converses with elderly and AD patients through truly natural speechpowered by axteam of trained human staffThe unique data sets that our solution enables us to gather at commercial scale will be leveraged in this SBIR project to develop an automatic speech recognitionASRand natural language processingNLPengine that is best in class for AD applicationsimproving the commercial scalability of our avatars by reducing our dependence on human staffwhile serving as a new AI platform for enabling the next generation of ADfocusedconversational SARs