SBIR Phase I: Holistic System for Comprehensive Student Assessment

The broader impact of this Small Business Innovation Research (SBIR) Phase I project is to help educators meet the needs of all their students by leveraging AI and Natural Language Processing (NLP) tools to examine robust sets of student learning data including quantitative and qualitative samples such as essays, written assignments, lab reports, and reflections, to determine student progress based on specific standards and competencies for more holistic and comprehensive assessment of student learning. The real-time, detailed analysis of student learning through qualitative and quantitative data analysis enables educators and administrators to understand how each learner, class, grade, and school is progressing in their learning. By contrast to more summative, end-of-course or end-of-year assessments which offer limited or delayed insights on student learning, this project provides educators and learners with access to deep analysis of student learning to make systemic course corrections and enable teachers to identify which standards and skills student's have been mastered and which need additional support in support of a more holistic approach to assessment and learning in primary and secondary education. This Small Business Innovation Research (SBIR) Phase I project will investigate the effects targeted large language model (LLM) fine-tuning using parameter-efficient fine-tuning (PEFT) and natural language processing (NLP) and infinite-context LLM based natural language generation (NLG) on qualitative and quantitative assessments of learners in grades 5-12. This research goal addresses, first, the problem that NLG is being used to generate feedback and content without targeted fine-tuning. There is an opportunity to use PEFT to allow for rapid, individualized NLG. Second, assessment relies on grades and tests that may not capture learning as robustly as necessary for a more holistic assessment mechanisms to make rapid and real-time shifts and provide comprehsive feedback. The technological innovation will use infinite context LLM pipelines and NLP techniques to allow teachers and administrators to gain a more complete view of students’ learning over time. This technical innovation will be paired with discourse analysis of collaborating educators and administrators to investigate effects of these novel NLP and NLG technologies on student learning over time. It is anticipated the intervention will provide educators with much greater visibility into distinct learning paths and provide timely feedback to improve K12 education. This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.