ENLIV-N: Effective Natural Language Interface for Unmanned Vehicle Navigation

Scientific Systems Company, Inc. (SSCI), in collaboration with Spatio Systems LLC and Prof. Bennett at Olin College, proposes to design and implement ENLIV-N, an Effective Natural Language Interface for Vehicle Navigation. This multi-modal user interface will translate natural language directions and gesture commands into vehicle waypoints, and coordinate feedback to the operator via video display and synthesized voice. Language understanding will be provided by Spatio's DUNE system, which uses corpora of real-world directions to train a novel phrase-based parsing approach that is uniquely robust to disfluencies. DUNE automatically augments its vocabulary using tagged databases (currently Flickr), and has been tested successfully on two ground robots and an aerial robot in indoor environments. Dr. Hsiao at SSCI will leverage his Ph.D. dissertation work on language-integrated robotic planning systems to provide multimodal integration and planning for the ENLIV-N system. Olin's vehicle from the DARPA Urban Challenge will serve as the autonomous test platform. Phase I work will focus on adapting DUNE in simulation to a mission of operational relevance, while designing the gesture, vehicle, and planning elements. At the end of Phase II, we will demonstrate our approach by commanding a real-world autonomous ground vehicle with waypoint navigation and obstacle avoidance.