Enhanced Text Analytics Using Lifted Probabilistic Inference Algorithms

ABSTRACT: LVI proposes developing an advanced framework of lifted probabilistic inference algorithms for enhancing the scaling and accuracy of text analytics. In Phase I, LVI explored the scalability of various lifted inference techniques for utilizing Markov Logic Networks (MLN) in the Tuffy software package. Phase I also included investigation and demonstration of DeepDive, a scalable, high-performance inference and learning engine for text analytics. These techniques were applied for automated knowledge base construction from free text, using abductive reasoning for optimal updates to the knowledge base. The MLN used unsupervised joint inferencing to combine record segmentation, co-reference resolution, and entity resolution in a single process, as opposed to a pipelined approach. The Phase II end-to-end prototype will be developed to perform text analytics over an information repository using the optimized joint inference technique. The prototype capabilities including joint inference over cross-document and multiple knowledge bases will be demonstrated through, for example, answering specific queries without considering the entire model and/or the entire evidence. "Distance supervision" and the Stanford Dependency Parser for NLP will be used to leverage external data sources for entity identification. Collaborating with a large financial institution MLN will be developed for entity recognition, relationship discovery and classification. ; BENEFIT: The algorithms for Lifted inference have a variety of applications. They include Social networks, object recognition, link prediction, activity recognition, model counting, bio-medical applications and relational reasoning and learning. Fundamental building block to improve current capabilities.