Peer-to-Peer Embedded Human Robot Interaction

The objective of this proposal is develop a concept for embedded peer-to-peer human/robot interaction that will allow a robot to work side-by-side with a person or a group of people in contrast to more traditional deployments of robotic hardware which keep the robots and humans very far from each other. MobileRobots Inc and Carnegie Mellon University (CMU) will exploit advances in dynamic perception and probabilistic reasoning techniques to develop a concept for robust human robot interaction (HRI) that will provide a robot with the necessary situation awareness to achieve a high level of peer-to-peer interaction. In the proposed concept, humans will interact with the robot in a natural fashion using speech, gestures, and verbal dialog. To facilitate such understanding of the human's activities and communicative acts, we will exploit the state of the art in computer vision techniques (both monocular and stereo) to detect and recognize the presence of humans, monitor and understand their gestures, and track their activities over time. This multi-modal situation awareness will allow the robot to monitor the actions and speech of a human in order to interactively learn to execute a state sequence for rudimentary joint plans with the human.