COMPUTER-BASED ITEM POOL EVALUATION SOFTWARE

SUBSTANTIAL IMPROVEMENTS IN TEST EFFICIENCY RESULTING FROM THE USE OF ITEM RESPONSE THEORY AND COMPUTERIZED TESTING MAY BE COMPROMISED IF DUPLICATE OR SUBSTANTIALLY SIMILAR TEST ITEMS APPEAR IN ITEM POOLS. THE PROPOSED PROJECT WILL DESIGN, IMPLEMENT, AND EVALUATE SEVERAL PROCEDURES FOR DETECTING SUBSTANTIALLY SIMILAR ITEMS IN AN ITEM POOL. THESE PROCEDURES WILL RANGE FROM SIMPLE ELEMENT AND TOKEN MATCHING PROCEDURES TO PROCEDURES MATCHING ELEMENTS OF SYNTACTIC AND SEMANTIC PARSES OF THE ITEMS. THE EVALUATION WILL COMPARE THE EFFECTIVENESS OF THE PROCEDURES WHEN APPLIED TO REPRESENTATIVE CAT-ASVAB ITEM BANKS. ADDITIONALLY, THE UTILITY OF THE INTERMEDIATE STRUCTURES AND STRUCTURAL PARAMETERS IN GENERATING NEW ITEMS AND PREDICTING RESPONSE-MODEL ITEM PARAMETERS WILL BE INVESTIGATED. PHASE I WILL RESULT IN PROGRAMS TO DETECT DUPLICATE ITEMS. PHASE II WILL REFINE THE PROGRAMS FOR LARGE-SCALE PRODUCTION USE AND DEVELOP THE TECHNIQUES OF ITEM GENERATION AND PARAMETER PREDICTION, SHOULD PHASE I SHOW THE LATTER TO BE FEASIBLE.