New CAT Research Tool:IRT/CAT Software for Secure Web and IVR Survey Administration

DESCRIPTION (provided by applicant): Computer Adaptive Testing (CAT) provides a new mechanism for assessing patient-reported outcomes. Using CAT can: (a) increase measurement precision, leading to a decrease in the number of participants necessary to achieve statistically significant results, (b) decrease test burden on respondents, compared to administration of fixed length surveys, thereby increasing the potential number of participants willing to participate in research, and (c) provide a common metric for assessing the effectiveness of varying treatment modalities. All of these capabilities will speed the transfer of clinical intervention effectiveness research into clinical care. The highly successful NIH Patient Reported Outcomes Measurement Information System (PROMIS) roadmap initiative has developed a diverse set of well-validated IRT-based clinical symptoms and functioning item banks as well as a web site for administering the items as fixed-length and CAT surveys. However, the PROMIS Assessment Center is not 21-CFR-Part-11 compliant, and therefore cannot be used in FDA clinical trials. In addition, it does not currently permit the creation of Interactive Voice Response (IVR) surveys, nor does it allow clients to store data privately or run surveys on a corporate website. In fact, at present, no software package is available that can be deployed on a corporate server to administer web or IVR surveys using CAT algorithms. Unless commercial survey software with 21-CFR-Part-11 compliant CAT functionality is developed, the benefits of the PROMIS initiative may not be realized. The overarching goal of this proposal is to create a CAT software module ( SmartCATTM ), using FDA- standard 21-CFR-Part-11 Best Practices, that can be integrated into existing survey software products. We will develop SmartCATTM with assistance from Seung Choi, PhD, and Steve Reise, PhD, two outstanding IRT-CAT researchers. SmartCATTM will be tested via formal design review, manual and automated operational testing, validation of actual algorithmic operations against existing standards, and two full rounds of field testing with potential customers. SmartCATTM will also be integrated into two existing TeleSage software packages: (1) SmartQ-WebTM for web survey administration and (2) SmartQ-IVRTM for automated telephone survey administration, as well as software packages created by Interviewing Services of America and SurveyGizmo, two large survey research companies with whom we have partnered for this project. Once successful integration into all four of these programs has been demonstrated, we will develop and integrate advanced CAT features, e.g., content balancing and enemy item control. The resulting TeleSage SmartCATTM and SmartQ survey suite will provide tremendous flexibility for CAT survey administration, thereby promoting greater measurement precision while decreasing respondent burden. For these reasons and because the proposed software package will make widespread private sector use of the existing PROMIS item banks possible, we believe that this is likely to be a high impact project. PUBLIC HEALTH RELEVANCE: The commercial suite of CAT survey administration software products proposed for development in this application, namely SmartCATTM, SmartQ-Web-CATTM, and SmartQ-IVR-CATTM, will afford significant reductions in the time required for patients to complete symptom and functioning assessments, thereby reducing the test burden on patient respondents, without significant loss of measurement precision. Making CAT survey software products available to a wider audience, including commercial entities, will speed the translation of research into clinical care and facilitate the realization of the goals of the NIH PROMIS roadmap initiative.