Lexicalized Multi-Modal Inference for Natural Grounding (LEMMING)

LEMMING will represent and implement state of the art inference for establishing and maintaining common ground within SIFT's ELEXIR system (a scalable probabilistic framework for reasoning about action).  The core result of LEMMING will be to show that ELEXIR is able to efficiently encode the specialized forms of reasoning required for common ground, and scale its reasoning to address the complexity of real common ground problems.  ELEXIR performs probabilistic reasoning about actions and mental states using MonteCarlo Tree Search (MCTS) and weighted model counting over models built from first-order, lexicalized action grammars.  As such it provides three benefits over prior and existing reasoning approaches for common ground. First, using MCTS will provide LEMMING with tight control over the computational resources used. Second, weighted model counting will enable LEMMING to perform probabilistic reasoning with all its attendant advantages. Third, first-order lexicalized action grammars both support the kind of rich models used in prior academic research (but missing from state of the art systems) and will allow LEMMING to efficiently direct its inferential search for common ground. All three of these features are critical to establish and maintain human-machine, multimodal common ground.