# ContactDexNet: Multi-fingered Robotic Hand Grasping in Cluttered Environments through Hand-object Contact Semantic Mapping

arXiv: https://arxiv.org/abs/2404.08844 · HTML: https://mlsanity.com/paper/2404.08844/

## Key facts

- Cited by 11 papers in the map (data through 14 Oct 2026).
- Builds on 30 of its 55 references that are in the map.
- First posted on arXiv 12 Apr 2024.

## Builds on (most cited first, 30 of 30)

- [PointNet++: Deep Hierarchical Feature Learning on Point Sets in a Metric Space](https://mlsanity.com/paper/1706.02413/) (2017)
- [GRAB: A Dataset of Whole-Body Human Grasping of Objects](https://mlsanity.com/paper/2008.11200/) (2020)
- [DexYCB: A Benchmark for Capturing Hand Grasping of Objects](https://mlsanity.com/paper/2104.04631/) (2021)
- [6-DOF GraspNet: Variational Grasp Generation for Object Manipulation](https://mlsanity.com/paper/1905.10520/) (2019)
- [Contact-GraspNet: Efficient 6-DoF Grasp Generation in Cluttered Scenes](https://mlsanity.com/paper/2103.14127/) (2021)
- [ContactPose: A Dataset of Grasps with Object Contact and Hand Pose](https://mlsanity.com/paper/2007.09545/) (2020)
- [Hand-Object Contact Consistency Reasoning for Human Grasps Generation](https://mlsanity.com/paper/2104.03304/) (2021)
- [DexGraspNet: A Large-Scale Robotic Dexterous Grasp Dataset for General Objects Based on Simulation](https://mlsanity.com/paper/2210.02697/) (2022)
- [CPF: Learning a Contact Potential Field to Model the Hand-Object Interaction](https://mlsanity.com/paper/2012.00924/) (2020)
- [SE(3)-DiffusionFields: Learning smooth cost functions for joint grasp and motion optimization through diffusion](https://mlsanity.com/paper/2209.03855/) (2022)
- [Benchmarking in Manipulation Research: The YCB Object and Model Set and Benchmarking Protocols](https://mlsanity.com/paper/1502.03143/) (2015)
- [Volumetric Grasping Network: Real-time 6 DOF Grasp Detection in Clutter](https://mlsanity.com/paper/2101.01132/) (2021)
- [ContactGrasp: Functional Multi-finger Grasp Synthesis from Contact](https://mlsanity.com/paper/1904.03754/) (2019)
- [PointNetGPD: Detecting Grasp Configurations from Point Sets](https://mlsanity.com/paper/1809.06267/) (2018)
- [LEAP Hand: Low-Cost, Efficient, and Anthropomorphic Hand for Robot Learning](https://mlsanity.com/paper/2309.06440/) (2023)
- [GenDexGrasp: Generalizable Dexterous Grasping](https://mlsanity.com/paper/2210.00722/) (2022)
- [ContactGen: Generative Contact Modeling for Grasp Generation](https://mlsanity.com/paper/2310.03740/) (2023)
- [AffordPose: A Large-scale Dataset of Hand-Object Interactions with Affordance-driven Hand Pose](https://mlsanity.com/paper/2309.08942/) (2023)
- [Deep Differentiable Grasp Planner for High-DOF Grippers](https://mlsanity.com/paper/2002.01530/) (2020)
- [Fast-Grasp'D: Dexterous Multi-finger Grasp Generation Through Differentiable Simulation](https://mlsanity.com/paper/2306.08132/) (2023)
- [DDGC: Generative Deep Dexterous Grasping in Clutter](https://mlsanity.com/paper/2103.04783/) (2021)
- [Learning Diverse and Physically Feasible Dexterous Grasps with Generative Model and Bilevel Optimization](https://mlsanity.com/paper/2207.00195/) (2022)
- [Multi-FinGAN: Generative Coarse-To-Fine Sampling of Multi-Finger Grasps](https://mlsanity.com/paper/2012.09696/) (2020)
- [DexRepNet: Learning Dexterous Robotic Grasping Network with Geometric and Spatial Hand-Object Representations](https://mlsanity.com/paper/2303.09806/) (2023)
- [DVGG: Deep Variational Grasp Generation for Dextrous Manipulation](https://mlsanity.com/paper/2211.11154/) (2022)
- [GPR: Grasp Pose Refinement Network for Cluttered Scenes](https://mlsanity.com/paper/2105.08502/) (2021)
- [Pixel-Attentive Policy Gradient for Multi-Fingered Grasping in Cluttered Scenes](https://mlsanity.com/paper/1903.03227/) (2019)
- [Towards Precise Model-free Robotic Grasping with Sim-to-Real Transfer Learning](https://mlsanity.com/paper/2301.12249/) (2023)
- [A Collision-Aware Cable Grasping Method in Cluttered Environment](https://mlsanity.com/paper/2402.14498/) (2024)
- [Learning to Detect Multi-Modal Grasps for Dexterous Grasping in Dense Clutter](https://mlsanity.com/paper/2106.03919/) (2021)

## Built on by (most cited first, 11 of 11)

- [DexGraspNet 2.0: Learning Generative Dexterous Grasping in Large-scale Synthetic Cluttered Scenes](https://mlsanity.com/paper/2410.23004/) (2024)
- [ClearDepth: Enhanced Stereo Perception of Transparent Objects for Robotic Manipulation](https://mlsanity.com/paper/2409.08926/) (2024)
- [Don't Let Your Robot be Harmful: Responsible Robotic Manipulation via Safety-as-Policy](https://mlsanity.com/paper/2411.18289/) (2024)
- [DETACH: Cross-domain Learning for Long-Horizon Tasks via Mixture of Disentangled Experts](https://mlsanity.com/paper/2508.07842/) (2025)
- [From Flow to One Step: Real-Time Multi-Modal Trajectory Policies via Implicit Maximum Likelihood Estimation-based Distribution Distillation](https://mlsanity.com/paper/2603.09415/) (2026)
- [FUNCanon: Learning Pose-Aware Action Primitives via Functional Object Canonicalization for Generalizable Robotic Manipulation](https://mlsanity.com/paper/2509.19102/) (2025)
- [M4Diffuser: Multi-View Diffusion Policy with Manipulability-Aware Control for Robust Mobile Manipulation](https://mlsanity.com/paper/2509.14980/) (2025)
- [OmniDexVLG: Learning Dexterous Grasp Generation from Vision Language Model-Guided Grasp Semantics, Taxonomy and Functional Affordance](https://mlsanity.com/paper/2512.03874/) (2025)
- [DORA: Object Affordance-Guided Reinforcement Learning for Dexterous Robotic Manipulation](https://mlsanity.com/paper/2505.14819/) (2025)
- [Leveraging Extrinsic Dexterity for Occluded Grasping on Grasp Constraining Walls](https://mlsanity.com/paper/2507.14721/) (2025)
- [D3Grasp: Diverse and Deformable Dexterous Grasping for General Objects](https://mlsanity.com/paper/2509.19892/) (2025)

## Abstract (arXiv)

The deep learning models has significantly advanced dexterous manipulation techniques for multi-fingered hand grasping. However, the contact information-guided grasping in cluttered environments remains largely underexplored. To address this gap, we have developed a method for generating multi-fingered hand grasp samples in cluttered settings through contact semantic map. We introduce a contact semantic conditional variational autoencoder network (CoSe-CVAE) for creating comprehensive contact semantic map from object point cloud. We utilize grasp detection method to estimate hand grasp poses from the contact semantic map. Finally, an unified grasp evaluation model PointNetGPD++ is designed to assess grasp quality and collision probability, substantially improving the reliability of identifying optimal grasps in cluttered scenarios. Our grasp generation method has demonstrated remarkable success, outperforming state-of-the-art methods by at least 4.65% with 81.0% average grasping success rate in real-world single-object environment and 75.3% grasping success rate in cluttered scenes. We also proposed the multi-modal multi-fingered grasping dataset generation method. Our multi-fingered hand grasping dataset outperforms previous datasets in scene diversity, modality diversity. The dataset, code and supplementary materials can be found at https://sites.google.com/view/contact-dexnet.