Advanced Indexing and Search for Efficient Information Discovery

ABSTRACT:In this project, we are developing a semantic search system, ActiveSearch, which builds upon the extraction capabilities of natural language processing engines, and also allows additional ontological information to be used for search. ActiveSearch focuses on two key requirements. The first is that an analyst must be able to quickly and easily convey the types of information that should be targeted. To achieve this capability, we build on recent advances in automatic query expansion and active learning. Our approach allows queries to be expanded via a massive ontology, and then reformulated based on user feedback, so that the system can accurately recognize the targeted content. The second requirement is that the system be fast enough to respond to a user in real-time. To achieve this capability, we develop a novel approach that employs a sparse bit set representation for extremely fast query processing, together with a distributed architecture.BENEFIT:Todays analysts are faced an enormous number of documents and yet it can be difficult for them to find the information they need. Search engines can be ineffective for researching topics that cannot be easily be described in a few keywords. On the other end of the spectrum, text extraction engines can identify targeted facts quite precisely, but they need to be trained and laboriously tuned for each task. ActiveSearch represents a middle ground, allowing analysts to pose hard informational queries that can be rapidly tuned for a given task. ActiveSearch is a research engine designed to save analysts significant time when sifting through large document corpuses. The approach will support military and intelligence analysts, as well as commercial partners that aggregate data in a variety of markets, including cyber-intelligence and background screening.