Semi-Autonomous Manipulator Control Using 2D and 3D Scene Recognition

Current mobile manipulators require extensive operator involvement to perform even simple tasks, such as opening a door, picking up an object, or emptying a container. “Fly-the-gripper” algorithms have been implemented to free the user from specifying joint velocities when a desired x, y, z gripper path is desired (Berkemeier, Poulson, et al. 2008), (Johnston, et al. 2008). Further, “point-and click” or “click-and-go” functionality allows the user to choose an object in a display and command the gripper to automatically move to the vicinity of the selected point. While this saves the user some trouble, the significantly more difficult task of grasping an object, picking it up, and manipulating it must still be performed manually. These manual operations are performed using a joystick, puck, mouse, or similar input device which requires concentration and hand dexterity and coordination. Often, mobile manipulators are used in difficult situations where technicians are under pressure to finish their tasks quickly in order to preserve human lives (possibly their own). Thus, it is crucial that semi-autonomous robot control algorithms be developed. For maximum benefit, the methods should apply to both existing and future platforms. For existing platforms, a “retro-fit kit” would add semi-autonomous manipulation capabilities to the thousands of mobile manipulators already in use. Future platforms, on the other hand, could be designed with suitable hardware already in place to implement the necessary algorithms developed in the proposed work for semi-autonomous manipulator control.