Adapting Information Extraction Technology to Computer-Mediated, Dynamic Text Data

Applications for synchronous computer-mediated communication, i.e. chat, like instant messaging and chatroom channels, are playing an ever-increasing role in military operations. Information extraction (IE) from chat could provide great value to dynamic targeting and other processes that depend on battlespace awareness. However, no previous work exists on performing IE on chat data, and only limited research has been done on the linguistic differences between chat, spoken dialog, and written text. Chat likely poses several challenges for standard IE methods developed for heavily-edited written text, including: (i) usage of punctuation, orthography, spelling, and grammar that differs from the written standard; (ii) high frequency of context-dependent and anaphoric linguistic forms; (iii) complex discourse structures. In Phase 1 of this project we will perform a systematic corpus study of task-oriented chat in order to (1) determine which properties of chat will be the most significant barriers to high accuracy IE; (2) assess which areas and techniques are likely to have the lowest cost-benefit ratio in developing a chat-based IE engine; and (3) develop initial design requirements for such an engine. This work will provide a solid foundation for the implementation of a prototype IE system capable of processing chat data in Phase 2.