Visually Guided Robotic Hand / Eye Coordination

There are many mundane tasks that humans – even children – perform easily, yet are unattainable by current robotic systems. These include dexterous manipulation in unstructured environments, where the exact location and characteristics of the objects are not known a priori, such as picking fruit or doing dishes. Recent years have seen many advances in both robotic manipulation and machine vision. Robots pick-and-place electronic components faster than the eye can see, and vision systems routinely inspect products hundreds of times per minute. Despite this, robotic and vision systems remain highly specialized in tightly controlled environments. We need a way to integrate these separate capabilities into a single closed-loop system. We will produce dexterous hand-eye coordination by focusing on four major subsystems: A vision system capable of identifying and localizing target objects in real-time, a hand capable of manipulating the objects, an arm capable of positioning the hand to the desired location, and a control system able to coordinate the other systems. TRACLabs has experience with all four of these areas, and is capable of integrating them into a unified system. In fact, TRACLabs scientists have already done this for NASA’s humanoid Robonaut to perform tool-use tasks similar to those in the solicitation.