SBIR Phase II: Re-envisioning alt text for education through concurrent authoring and diagram design

The broader impact of this SBIR Phase II project is to create a pathway to success in STEM for all students, including those who are blind or have low vision (BLV), with a truly usable and effective description system for STEM diagrams. The lack of accessible accommodations creates roadblocks for BLV students in STEM education, leading many students to give up, even though they have equal potential as the general population for academic success. All web-based, non-decorative diagrams are required to include alternative (alt) text descriptions for use with screen readers. Current methods for making STEM visualizations accessible through alt text are not scalable or amenable for use with dynamic digital media. Visualizing STEM concepts through a linear string of text can be difficult for BLV students, because of the high cognitive load. Integrating an interactive method for querying specific information provides screen reader users a more personalized learning experience. This interactive system can be used by all students to make sense of STEM concepts, creating a use case where all users benefit from the inclusion of alt text. Commercialization will be from integration into existing learning platforms to meet accessibility requirements required of federally funded educational institutions. This Small Business Innovation Research Phase II project will create an expandable system for generating detailed and standardized descriptions of STEM diagrams in interactive media. The goal is to provide real-time alt text generation for screen reader users (SRUs) and to create an inclusive and accessible communication method for all students in STEM courses. The alt text description engine provides a method for integrating visual information from STEM diagrams with large language models and the algorithms of current artificial intelligence (AI) platforms. The AI-driven learning assistant from this project guides SRUs in understanding complex alt text descriptions in a manner that best matches their skillsets and prior knowledge. The AI learning assistant also provides personalized contextual learning support for all users. The technical objectives include 1) Expanding the architecture to scale the alt text generation engine to other STEM subjects and readily integrate with AI platforms, 2) Implementing accessibility features to the suite of learning interactives, and 3) Developing a database of learning activities aligned to learning objectives. Usability studies with SRUs and instructors will guide iterative product development. Research studies in classrooms will demonstrate the promise of outcomes to enhance commercial success for the suite of accessible learning interactives. This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.