SBIR Phase II: A virtual role-playing simulation for social emotional learning using artificially intelligent characters and crowdsourcing

This SBIR Phase II project will develop novel 3D web and mobile role-playing simulations that can improve social skills in children. Virtual characters participate in nuanced conversations with students, powering a transformational experience for practicing and assessing communication skills. Rigorous evaluations have demonstrated the feasibility and efficacy of teaching perspective-taking and social skills through simulated role-playing, beneficial to students struggling with autism spectrum disorder (ASD), attention deficit hyperactivity disorder (ADHD), or bullying. These simulations will disrupt the multi-billion dollar market for social emotional learning (SEL) by allowing students to learn and practice social skills as they would with teachers or therapists, but at a fraction of the cost. Current solutions for practicing and assessing social skills rely on face-to-face interaction, which is costly and difficult to scale. Producing socially rich, open-ended simulations has previously been infeasible, due to technical challenges required to replicate the complexity of human language. This project overcomes these barriers using an innovative technology with the potential to revolutionize how people learn and practice social skills by delivering real-time personalized feedback at scale. Future applications of the social behavior capture (So-Cap) technology that underlies this project range from contextually aware intelligent personal assistants to socially aware robotics. This project's unprecedented ability to respond authentically to open-ended natural language input relies on an innovative crowdsourced approach to artificial intelligence (AI), which imbues machines with the ability to understand dialogue in context and engage in extended conversations covering multiple topics. Individualized responses are tailored to user input by drawing from a massive database of recorded human dialogue, captured from online role-playing. The process for mining meaningful patterns from this data also employs crowdsourcing, relying on non-experts hired online to cluster and annotate words, utterances, and events. A computational model formed by these patterns powers a real-time conversational engine, which selects responses by combining AI techniques including plan recognition and case-based planning. The goal of this research is to pioneer a practical, repeatable process for democratizing the production of social simulations, minimizing cost while maximizing societal impact by providing social skills practice at scale.