Modeling Topic Pragmatics for Improved First Story Detection

We propose an innovative combination of machine learning techniques and pragmatics modeling in pursuit of high accuracy, domain-and language-independent first story detection and topic tracking. Our Topic-Oriented Pragmatics and Invariant Chaining (TOPIC)system will exploit a diverse variety of pragmatic features identified in our Pragmatics Framework that structure stories and topics as they evolve from initial events to final text documents. Central to our pragmatics-based approach is the idea thatmultiple perspectives on story novelty can overcome errors made by any single perspective. Committee-based methods arbitrate these multiple classification viewpoints. Category models are constructed semi-automatically under user guidance to capturestereotypically co-occurring activities within broad categories of topics. These models are used to generate predictions about the future content of news stories and assist in customizing the system to a particular task domain. Scalability is ensuredthrough a combination of techniques, including temporal feature compression and granularity control. Our Phase I research and development of a proof-of-concept prototype demonstrated the feasibility and utility of TOPIC's pragmatics-based approach andprovides a solid foundation for a scalable, fully functional Phase II implementation of a first story detection and topic tracking system.