SBIR Phase I: Grading 10x Faster with AI Assistance

This SBIR Phase I project will show feasibility of a method for instant and accurate grading of student answers to a previously unseen question, after observing an instructor grade no more than 10% of the answers to that question. This project will use instructor-defined grading rubrics, which ensure consistency and provide helpful feedback to the student. Initial work will evaluate the method on multiple choice and fill-in-the-blank questions from a variety of Science, Technology, Engineering, and Math (STEM) subjects. The same method will be general enough to allow extension to short-answer and diagram questions. This work is important because although constructed-response questions are far more effective at assessing student knowledge and guiding student learning, they are significantly more time-consuming and onerous to grade than multiple-choice questions, and are therefore underused in many courses. Giving instructors the ability to grade constructed-response questions 10 times faster will increase the prevalence of this type of assessment and thereby improve STEM education outcomes. It will also increase educator effectiveness, because time that instructors currently spend grading can be better spent in personalized interaction with students. This project will apply and extend recently developed deep neural network methods for few-shot learning to the task of using a small sample of graded student answers to automatically grade the rest. Deep neural networks have recently emerged as the best approach to most machine learning problems. However, unlike humans, who can often learn a new concept after just a few examples, deep neural networks typically need thousands of examples to do the same. Recent developments have enabled deep neural networks to learn new concepts from just a few examples, after seeing millions of examples of other concepts. The first objective of this project is to evaluate these methods using millions of answers to a range of multiple-choice and fill-in-the-blank questions from a diverse set of STEM subjects graded on an existing service. This few-shot learning problem is significantly more complex than problems studied in the research literature. The second objective is to extend these methods, and to develop new methods, in order to reach human-level accuracy after observing the grading of no more than 10% of student answers. The last objective is to ensure that the proposed system is useful to real instructors by implementing a prototype user interface.