Holistic Grasping: Affordances, Grasp Semantics, Task Constraints

Sammanfattning: Most of us perform grasping actions over a thousand times per day without giving it much consideration, be it from driving to drinking coffee. Learning robots the same ease when it comes to grasping has been a goal for the robotics research community for decades.The reason for the slow progress lays mainly in the inferiority of the robot sensorimotor system. Robotic grippers are often non-compliant, lack the degrees of freedom of human hands, and haptic sensors are rudimentary involving significantly less resolution and sensitivity than in humans.Research has therefore focused on engineering solutions that center on the stability of the grasp. This involves specifying complex functions and search strategies detailing the interaction between the digits of the robot and the surface of the object. Given the amount of variation in materials, shapes, and ability to deform it seems infeasible to analytically formulate such a gripper-to-shape mapping. Many researchers have instead looked to data-driven methods for learning the gripper-to-shape mapping as does this thesis.Humans obviously have a similar mapping capability. However, how we grasp an object is determined foremost by what we are going to do with the object. We have priors on task, material, and the dynamics of objects that help guide the grasping process. We also have a deeper understanding of how shape and material relate to our own embodiment.We tie all these aspects together: our understanding of what an object can be used for, how that affects our interaction with it, and how our hand can form to achieve the goal of the manipulation. For us humans grasping is not just a gripper-to-shape mapping it is a holistic process where all parts of the chain matters to the outcome. The focus of this thesis is thus on how to incorporate such a holistic process into robotic grasp planning.  We will address the holistic grasping process through three jointly connected modules. The first is affordance detection and learning to infer the common parts for objects that afford an action, a form of conceptualization of the affordance categories. The second is learning grasp semantics, how shape relates to the gripper configuration. And finally the third is to learn how task constrains the grasping process.We will explore these three parts through the concept of similarity. This translates directly into the idea that we should learn a representation that puts similar types of the entities that we are describing, that is, objects, grasps, and tasks, close to each other in space. We will show that the idea of similarity based representations will help the robot reason about which parts of an object is important for affordance inference, which grasps and tasks are similar, and how the categories relate to each other. Finally, the similarity-based approach will help us tie all parts together in the conceptual demonstration of how a holistic grasping process might be realized.