Adapting Information Extraction Technology to Computer-Mediated, Dynamic Text Data

The goal of this project is to develop an information extraction (IE) application that can process military chat data. This application will be developed as a version of Janya's existing IE engine, Semantex, with a new chat-specific architecture. This architecture will allow automated monitoring of streaming real-time chat conversation and rapid generation of alerts when objects of interest are mentioned. The flexibility and modularity of Semantex's design will facilitate the rapid development these capabilities. In addition, Semantex already contains extensible solutions for domain customization and processing noisy data useful for chat data. Nonetheless, the linguistic complexities posed by chat mean the problem of high-level IE (for example, combining multiple entity mentions into a single entity profile) is extremely hard. Better, more complete extraction will require conversation-level processing, as important facts about entities of interest may only be mentioned using anaphoric constructions that require decoding at the conversation level. As such, the bulk of the research effort in Phase II will be directed at solving those problems. Because chat data is of ever-increasing importance in the military domain, developing a chat IE application is an opportunity to further enhance the advantages of this rapid but non-intrusive form of communication.