Automating Tabular Data Extraction to Accelerate Evidence Synthesis and Systematic Literature Review

PROJECT SUMMARY Systematic Literature Reviews (SLRs) provide the safety and efficacy data that inform decision-making throughout the healthcare ecosystem, from regulatory approval and Health Technology Assessment (HTA) for reimbursement to society guidelines and physician decision-making. Despite serving this crucial function, the current methods used to perform SLRs are highly work-intensive, with the typical review completed manually in Excel–and taking an average of $141,000 and 18 months. This has led to a deficiency in coverage of key research questions, with only 10-17% of clinical topics having an up-to-date SLR. While some workflow tools exist for specific SLR steps, no conventional SLR tools provide either all-in-one Search, Screening, and Extraction of data or artificial intelligence (AI) assistance at any step. Both the Data and Informatics Working Group of the NIH and the scientific community have recognized that the bottleneck in clinical science is in evidence synthesis and a novel software ecosystem is needed to solve this pressing problem. Nested Knowledge was founded in 2018 to build a platform for collecting, analyzing, and presenting data. The platform is divided into AutoLit, a full workflow for ‘living’, updatable, AI-assisted Search, Screening, and Extraction from full texts, and Synthesis, an interactive data-visualization hub with qualitative SLR findings and insights as well as automated Network Meta-analytical statistics. Since launching, the platform has been adopted by leading research institutions, 6 of the top 10 HTA providers, and the US standard-setting body for HTA, the Institute for Clinical and Economic Review (ICER). Nested Knowledge’s software provides every step demanded by SLR professionals, and with the advent of Large Language Models (LLMs), textual extraction has been proven feasible. In fact, we have built and launched a biomedical-specific LLM for extracting from full texts. However, a key step remains entirely manual: extraction of Interventions, Data Elements, and Timepoints from tables in published studies. This vital and work-intensive step has been recognized as the next frontier of AI-assisted SLR, and in this Phase I SBIR proposal, we propose to build, test, and launch an LLM-based tool for extracting data from tables, which can then be fed into existing automated Network Meta-analytical tools in the platform. If successful, this SBIR grant will provide the research community with a biomedical-specific tabular extraction LLM, integrated into an existing SLR workflow with a human oversight module. This project will also quantify the performance of this LLM to enable iterative improvement and support both commercialization of this tool and further research integrating text, table, and figure extraction to move closer to the ultimate goal of near- or fully-automated synthesis of data from published clinical studies.