SBIR Phase I: Measure What We Treasure: Developing a Structural Cognitive Analytics and Assessment (SCAA) Technology

This SBIR Phase I project will address the pedagogical and evaluative problems caused by our current inability to measure core outcomes beyond information recall. The proposed innovation makes important educational outcomes such as deep understanding, cognition (thinking skills) and awareness of thinking (metacognition) measurable. Reliable assessment of these outcomes has broad significance to education and society writ large, as many employers struggle with skills gaps between high school and college graduates and the professional, personal and societal demands on adults in the 21st Century. This research combines cognitive science, epistemology, systems science, and complex systems with new developments in artificial intelligence, including machine learning and neural networks, and aligns well with the NSF's mission to promote progress in science and advance national prosperity. This project not only has the potential to impact core tenets of educational practice - including how teachers teach, how learners learn, and how we measure and understanding knowledge - but also may have impact on science through increased ability to map and analyze patterns and common structures in knowledge and generally for Americans to increase their developmental skills and abilities. Beginning in an educational market valued at $8-15 billion, this project has the potential to catalyze significant job creation and have significant commercial impact across a range of related industries. This SBIR Phase I project introduces a visual grammar for mapping ideas in a canvas-based environment, and provides a neural-network based mechanism for quantitatively comparing expressions of complex ideas along many dimensions to facilitate a new approach to thinking, learning, and assessment. Maps of ideas will be tokenized and serialized and then fed through a recurrent neural network model to produce encoded numeric vector representations that are meaningfully comparable in their encoded form. Once in this form, vectors will be compared and evaluated for content and structure similarity, and insights offered in many dimensions: depth of detail, understanding of relationships, and numerous other meaningful measures. These vectors enable scalable, consistent, constructive assessment in a way not previously possible. In the mapping environment, teachers and students will create maps either on their own, or collaboratively together in real-time. As users create maps, we use the generated vectors to analyze both content and structure, and then prompt users to think about their subject matter from new perspectives. As a means of evaluation, teachers will create mapping activities for students to complete, and then student maps will be quantitatively compared to the standards and target maps which represent the complete understanding of the topic. This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.