Advanced Assessment to Accelerate Diagnostic Skill Acquisition, Phase II

Project Summary Today s clinical learning environments do not provide the level of deliberate practicedirect supervisionand rigorous assessment and feedback needed to develop diagnostic reasoning expertiseClinical performance assessment emphasizes learner evaluation over learner developmentlacks rigor and utility for developmental purposesand clinical teachers have expressed particular difficulty with diagnosing reasoning deficits for remediation purposesFurthermedical studentsdiagnostic reasoning does not improve over the course of clinical training and senior medical students have highly variable diagnostic performance that is often rated below expectations according to theory based and validated scoring criteriaIndependent practice does not necessarily enhance the context for clinical reasoningthe majority of physiciansmedical errors are thought to be diagnostic in natureWe propose to improve undergraduate medical education to minimize the time to clinical competency for first year residents through targeted diagnostic reasoning skill development thatintegrates basic science and clinical instructionprovides deliberate practice with structuredcase based learning opportunitiesandenables anytime anywhere learning that fits with the demanding schedules of most medical studentsSouthern Illinois University School of MedicineSIUSOMis a recognized leader in using performance based clinical competency exams to enhance reasoning skill acquisition among medical studentsThese exams feature clinical scenarios with standardized patients followed by diagnostic justification essays which require students to explicitly describe the thought process used to reach a final diagnosisThese essays are the most reliable method of assessing diagnostic strategies but are not in use in the majority of medical schoolsthough interest in improving diagnostic reasoning instruction and assessment during undergraduate medical education is widespreadBarriers to the widespread adoption of this approach arethe time consuming need to hand score each essayandthe difficulty in accurately and consistently identifying the causes of strategy failuresThis project will develop an application to provide automated scoring of diagnostic justification essaysidentification of the underlying causes of failure when students perform poorlyand feedback with instructional strategies for remediation specific to each deficitWe propose these specific aimsImprove reliability of human scoring of DXJ essaysExtend the automated scoring algorithmsAutomated reasoning failure categorization and remediationComplete the software development required for delivering the commercial productEvaluate predictive validity of automatically scored DXJ essaysThe proposed product represents a significant shift in undergraduate medical training and through Phase III dissemination will address a critical gap between education and practice in academic medicine