Storytelling for Causal Data Mining

The intelligence gathering community needs automated data mining and discovery of causal relationships in unstructured textual documents. Harmonia proposes to fill this need via our Raconteur system. Raconteur can benefit an analyst by guiding them in automated discovery of unknown document linkages. Examples include grouping a set of terrorist attacks with similar methodologies and locations, discovering links between two individuals to expose a network of suspicious activities, or determining if two entities with little-known knowledge affect one another. Raconteur will be a web-based application that produces interactive visualizations allowing a user to construct and manipulate graphs showing chains of evidence and corroborating evidence that forms stories connecting two documents of interest. Our proposed work on Raconteur includes implementation of its analysis algorithms using parallel/distributed processing using the MapReduce framework. These techniques have been applied to small and large data sets, such as the Tactical Ground Reporting (TIGR) system on a classified dataset of 1.75 million reports. The results of these techniques can be used to quickly identify documents related to a topic and find the missing links between given reports. By finding such chains of evidence, an analyst can quickly determine unknown relationships between topics, persons, events, or places.