SWIFT ActiveScreener research and development of an intelligent web based document screening system

Project Summary More than systematic reviews are performed each year in the fields of environmental health and evidence based medicine with each review requiring on average between six months to one year of effort to complete In order to remain accurate systematic reviews require regular updates after their initial publication with most reviews out of date within five years In the screening phase of systematic review researchers use detailed inclusion exclusion criteria to decide whether each article in a set of candidate citations is relevant to the research question under consideration For each article considered a researcher reads the title and abstract and evaluates its content with respect to the prespecified criteria A typical review may require screening thousands or tens of thousands of articles in this manner Under the assumption that it takes a skilled reviewer seconds on average to screen a single abstract dual screening a set of abstracts may require between to hours of labor We have shown in previous work that automated machine learning methods for article prioritization can reduce by more than the human effort required to screen articles for inclusion in a systematic review Recently we have further extended these methods and packaged them into a web based collaborative systematic review software application called SWIFT Active Screener Active Screener has been used successfully to reduce the effort required to screen articles for systematic reviews conducted at a variety of organizations including the National Institute of Environmental Health Science NIEHS the United States Environmental Protection Agency EPA the United States Department of Agriculture USDA The Endocrine Disruption Exchange TEDX and the Evidence Based Toxicology Collaboration EBTC These early adopters have provided us with an abundance of useful data and user feedback and we have identified several areas where we can continue to improve our methods and software Our goal for the current proposal is to conduct additional research and development to make significant improvements to SWIFT Active Screener including several innovations that will be necessary for commercial success The research we propose encompasses three specific aims Investigate several improvements to statistical algorithms used for article prioritization and recall estimation We will explore promising avenues for further improving the performance of our existing algorithms and address critical technical issues that limit the applicability of our current methods Aim Improved Statistical Models Explore ways in which we can improve our models and methods to handle the scenario in which an existing systematic review is updated with new data several years after its initial publication Aim New Methods for Systematic Review Updates Investigate several questions related to scaling the system to support hundreds to thousands of simultaneous screeners Aim Software Engineering for Scalability Usability and Full Text Extraction Project Narrative Systematic review is a formal process used widely in evidence based medicine and environmental health research to identify assess and integrate the primary scientific literature with the goal of answering a specific targeted question in pursuit of the current scientific consensus By conducting research and development to build a web based collaborative systematic review software application that uses machine learning to prioritize documents for screening we will make an important contribution toward ongoing efforts to automate systematic review These efforts will serve to make systematic reviews both more efficient to produce and less expensive to maintain a result which will greatly accelerate the process by which scientific consensus is obtained in a variety of medical and health related disciplines having great public significance