SBIR Phase I: scite: A deep learning platform to identify confirming and refuting citations

This SBIR Phase I project is directed towards creating a resource that would enable non-experts to evaluate the validity of scientific claims. The current measures of scientific reliability - the number of times a scientific report is cited and the prestige of the journal that publishes it - are known to be inherently poor indicators of research quality but are still used for the lack of a better alternative. This deficiency costs the public, businesses, and governments billions of dollars and missed opportunities because using invalid claims to discover new medicines and technologies is unlikely to be successful. The proposed research and the resulting prototype will help a non-expert to evaluate scientific claims by automatically extracting classifying statements from scientific literature that provide supporting or contradicting evidence. If successful, the proposed innovation will encourage reliable research by vastly increasing the number of people who can evaluate it soundly, will enable individuals and organizations make better-informed decisions based on scientific evidence, and will help to educate both professionals and the public about available evidence outside their areas of expertise. This SBIR Phase I project will use the latest developments in deep learning and natural language processing to create a platform that can automatically analyze the veracity of scientific claims at previously unachievable speed and scale. This innovation will allow this project to analyze reported scientific claims, to collect the resulting information in a publicly available resource, and to provide this information to end users through a web-based intuitive user interface in an easy to grasp form. The specific aims of the proposed research are to make the accuracy of classifying statements that support, contradict, or merely mention a scientific claim approach that of a human, which will require this project to advance deep learning and text analysis even further, to increase the efficiency of extracting these statements from scientific literature, and to develop approaches for visualizing the results of veracity analysis in an intuitively graphical form. Despite rapid advances in text mining technology, analyzing the text of scientific reports is still challenging, both technically and logistically, due to their specific language and the peculiarities of report distribution. Accomplishing this project successfully will not only produce a highly needed tool but will also advance the emerging field of sentiment analysis of scientific citations. This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.