Rapid Event Analysis and Characterization via Targeted Output Refinement (REACTOR)

Publicly available information (PAI) such as social media can play a critical role in enabling DoD intelligence analysts to detect, understand, and maintain continuous situational awareness of emerging events involving national security, such as events relating to ongoing military conflicts around the globe. However, synthesizing these fragmented and often unreliable accounts into an accurate summary of events is challenging for analysts, who are faced with an overwhelming volume of data. The advent of generative large language models (LLMs) ľ which excel at integration and?abstractive summarization ľ therefore represents a potential game changer for analysts. However, LLMs are prone to confabulation errors and are subject to gaps and biases in their training data. To address these challenges, we propose to develop REACTOR, an application to empower DoD intelligence analysts with robust, LLM-based event analysis and summarization capabilities that create and maintain continuous situational awareness. REACTOR will build a custom application instance for each group of users, with options to specify 1) the domain and event types of interest, 2) the social media and news sources to use for event detection, analysis, and validation, and 3) the LLMs to deploy for event understanding. Once initialized, a REACTOR instance will continuously source social media data to summarize critical events.? REACTOR will also detect and correct sources of error and hallucination using techniques such as leveraging newer knowledge bases when generating summarizations.á