mlsanityData to 14 Oct 2026

Learning High-DOF Reaching-and-Grasping via Dynamic Representation of Gripper-Object Interaction

arXiv 2204.13998 ↗First posted 3 Apr 2022cs.RO cs.CV cs.GRcited by 24

Not on the map. Why?

Abstract (arXiv)

We approach the problem of high-DOF reaching-and-grasping via learning joint planning of grasp and motion with deep reinforcement learning. To resolve the sample efficiency issue in learning the high-dimensional and complex control of dexterous grasping, we propose an effective representation of grasping state characterizing the spatial interaction between the gripper and the target object. To represent gripper-object interaction, we adopt Interaction Bisector Surface (IBS) which is the Voronoi diagram between two close by 3D geometric objects and has been successfully applied in characterizing spatial relations between 3D objects. We found that IBS is surprisingly effective as a state representation since it well informs the fine-grained control of each finger with spatial relation against the target object. This novel grasp representation, together with several technical contributions including a fast IBS approximation, a novel vector-based reward and an effective training strategy, facilitate learning a strong control model of high-DOF grasping with good sample efficiency, dynamic adaptability, and cross-category generality. Experiments show that it generates high-quality dexterous grasp for complex shapes with smooth grasping motions.

Key facts. Cited by 24 papers in the map (data through 14 Oct 2026). Builds on 31 of its 74 references that are in the map. First posted on arXiv 3 Apr 2022.

Builds on

31 of 74 references.

References · 31

  1. 2016PointNet: Deep Learning on Point Sets for 3D Classification and Segmentationcited by 5,248
  2. 2018Soft Actor-Critic Algorithms and Applicationscited by 1,240
  3. 2018QT-Opt: Scalable Deep Reinforcement Learning for Vision-Based Robotic Manipulationcited by 743
  4. 2017Learning Complex Dexterous Manipulation with Deep Reinforcement Learning and Demonstrationscited by 715
  5. 2018Learning Dexterous In-Hand Manipulationcited by 687
  6. 2017Sim-to-Real Transfer of Robotic Control with Dynamics Randomizationcited by 672
  7. 2017Leveraging Demonstrations for Deep Reinforcement Learning on Robotics Problems with Sparse Rewardscited by 319
  8. 2016Learning Hand-Eye Coordination for Robotic Grasping with Deep Learning and Large-Scale Data Collectioncited by 316
  9. 2017Dex-Net 2.0: Deep Learning to Plan Robust Grasps with Synthetic Point Clouds and Analytic Grasp Metricscited by 264
  10. 2020ContactPose: A Dataset of Grasps with Object Contact and Hand Posecited by 183
  11. 2020Grasping Field: Learning Implicit Representations for Human Graspscited by 179
  12. 2018Sim-to-Real via Sim-to-Sim: Data-efficient Robotic Grasping via Randomized-to-Canonical Adaptation Networkscited by 174
19 more
  1. 2018Closing the Loop for Robotic Grasping: A Real-time, Generative Grasp Synthesis Approachcited by 120
  2. 2020Effective Diversity in Population Based Reinforcement Learningcited by 119
  3. 2019Grasping in the Wild:Learning 6DoF Closed-Loop Grasping from Low-Cost Demonstrationscited by 107
  4. 2013Data-Driven Grasp Synthesis - A Surveycited by 106
  5. 2018Jacquard: A Large Scale Dataset for Robotic Grasp Detectioncited by 87
  6. 2016Shape Completion Enabled Robotic Graspingcited by 79
  7. 2020Deep Differentiable Grasp Planner for High-DOF Gripperscited by 39
  8. 2017Multi-Task Domain Adaptation for Deep Learning of Instance Grasping from Simulationcited by 36
  9. 2016High precision grasp pose detection in dense cluttercited by 34
  10. 2018Deep Reinforcement Learning for Vision-Based Robotic Grasping: A Simulated Comparative Evaluation of Off-Policy Methodscited by 33
  11. 2017Learning a visuomotor controller for real world robotic grasping using simulated depth imagescited by 29
  12. 2019Learning Continuous 3D Reconstructions for Geometrically Aware Graspingcited by 29
  13. 2018Planning Multi-Fingered Grasps as Probabilistic Inference in a Learned Deep Networkcited by 19
  14. 2019Generating Grasp Poses for a High-DOF Gripper Using Neural Networkscited by 17
  15. 2019Manipulation Trajectory Optimization with Online Grasp Synthesis and Selectioncited by 17
  16. 2020AdaGrasp: Learning an Adaptive Gripper-Aware Grasping Policycited by 16
  17. 2016Understanding and Exploiting Object Interaction Landscapescited by 8
  18. 2019New Formulation of Mixed-Integer Conic Programming for Globally Optimal Grasp Planningcited by 1
  19. 2021Planning of Power Grasps Using Infinite Program Under Complementary Constraintscited by 1

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2022: 2; 2023: 12; 2024: 5; 2025: 2; 2026: 3.

24 papers.

Sort:

By area:

  1. 2023UniDexGrasp: Universal Robotic Dexterous Grasping via Learning Diverse Proposal Generation and Goal-Conditioned Policycited by 124
  2. 2023UniDexGrasp++: Improving Dexterous Grasping Policy Learning via Geometry-aware Curriculum and Iterative Generalist-Specialist Learningcited by 107
  3. 2024DiffH2O: Diffusion-Based Synthesis of Hand-Object Interactions from Textual Descriptionscited by 62
  4. 2024GraspXL: Generating Grasping Motions for Diverse Objects at Scalecited by 60
  5. 2022GenDexGrasp: Generalizable Dexterous Graspingcited by 56
  6. 2023CAMS: CAnonicalized Manipulation Spaces for Category-Level Functional Hand-Object Manipulation Synthesiscited by 56
  7. 2023ArtiGrasp: Physically Plausible Synthesis of Bi-Manual Dexterous Grasping and Articulationcited by 55
  8. 2024RealDex: Towards Human-like Grasping for Robotic Dexterous Handcited by 34
  9. 2024GigaHands: A Massive Annotated Dataset of Bimanual Hand Activitiescited by 33
  10. 2023GRIP: Generating Interaction Poses Using Spatial Cues and Latent Consistencycited by 27
  11. 2023GraspGF: Learning Score-based Grasping Primitive for Human-assisting Dexterous Graspingcited by 23
  12. 2023SparseDFF: Sparse-View Feature Distillation for One-Shot Dexterous Manipulationcited by 22
12 more
  1. 2023Grasp Multiple Objects with One Handcited by 18
  2. 2023HMDO: Markerless Multi-view Hand Manipulation Capture with Deformable Objectscited by 12
  3. 2023Learning to Transfer In-Hand Manipulations Using a Greedy Shape Curriculumcited by 5
  4. 2024Neural Attention Field: Emerging Point Relevance in 3D Scenes for One-Shot Dexterous Graspingcited by 5
  5. 2025G-DexGrasp: Generalizable Dexterous Grasping Synthesis Via Part-Aware Prior Retrieval and Prior-Assisted Generationcited by 5
  6. 2022NIFT: Neural Interaction Field and Template for Object Manipulationcited by 4
  7. 2023Reconstructing 3D Human Pose from RGB-D Data with Occlusionscited by 3
  8. 2026UniCross: Unified Cross-Skill Dexterous Manipulation Synthesiscited by 2
  9. 2023DMFC-GraspNet: Differentiable Multi-Fingered Robotic Grasp Generation in Cluttered Scenescited by 1
  10. 2026Adaptive Vision-Language Grasping via Composable Foundation Priors and Generalizable Grasp Synthesiscited by 1
  11. 2025PartHOI: Part-based Hand-Object Interaction Transfer via Generalized Cylinderscited by 0
  12. 2026Grasp2Twist: Learning Bimanual Dexterous Jar Opening by Reinforcement Learningcited by 0