# Learning 6-DoF Fine-grained Grasp Detection Based on Part Affordance Grounding

arXiv: https://arxiv.org/abs/2301.11564 · HTML: https://mlsanity.com/paper/2301.11564/

## Key facts

- Cited by 17 papers in the map (data through 14 Oct 2026).
- Builds on 39 of its 102 references that are in the map.
- First posted on arXiv 27 Jan 2023.

## Builds on (most cited first, 30 of 39)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformer](https://mlsanity.com/paper/1910.10683/) (2019)
- [PointNet: Deep Learning on Point Sets for 3D Classification and Segmentation](https://mlsanity.com/paper/1612.00593/) (2016)
- [PointNet++: Deep Hierarchical Feature Learning on Point Sets in a Metric Space](https://mlsanity.com/paper/1706.02413/) (2017)
- [ShapeNet: An Information-Rich 3D Model Repository](https://mlsanity.com/paper/1512.03012/) (2015)
- [Do As I Can, Not As I Say: Grounding Language in Robotic Affordances](https://mlsanity.com/paper/2204.01691/) (2022)
- [PartNet: A Large-scale Benchmark for Fine-grained and Hierarchical Part-level 3D Object Understanding](https://mlsanity.com/paper/1812.02713/) (2018)
- [ScanRefer: 3D Object Localization in RGB-D Scans using Natural Language](https://mlsanity.com/paper/1912.08830/) (2019)
- [Dex-Net 2.0: Deep Learning to Plan Robust Grasps with Synthetic Point Clouds and Analytic Grasp Metrics](https://mlsanity.com/paper/1703.09312/) (2017)
- [COMET-ATOMIC 2020: On Symbolic and Neural Commonsense Knowledge Graphs](https://mlsanity.com/paper/2010.05953/) (2020)
- [Contact-GraspNet: Efficient 6-DoF Grasp Generation in Cluttered Scenes](https://mlsanity.com/paper/2103.14127/) (2021)
- [AnyGrasp: Robust and Efficient Grasp Perception in Spatial and Temporal Domains](https://mlsanity.com/paper/2212.08333/) (2022)
- [Closing the Loop for Robotic Grasping: A Real-time, Generative Grasp Synthesis Approach](https://mlsanity.com/paper/1804.05172/) (2018)
- [ACRONYM: A Large-Scale Grasp Dataset Based on Simulation](https://mlsanity.com/paper/2011.09584/) (2020)
- [Deep Learning for Detecting Robotic Grasps](https://mlsanity.com/paper/1301.3592/) (2013)
- [Grasp Pose Detection in Point Clouds](https://mlsanity.com/paper/1706.09911/) (2017)
- [3D AffordanceNet: A Benchmark for Visual Object Affordance Understanding](https://mlsanity.com/paper/2103.16397/) (2021)
- [Jacquard: A Large Scale Dataset for Robotic Grasp Detection](https://mlsanity.com/paper/1803.11469/) (2018)
- [Learning Dexterous Grasping with Object-Centric Visual Affordances](https://mlsanity.com/paper/2009.01439/) (2020)
- [PointNetGPD: Detecting Grasp Configurations from Point Sets](https://mlsanity.com/paper/1809.06267/) (2018)
- [Deep Learning Approaches to Grasp Synthesis: A Review](https://mlsanity.com/paper/2207.02556/) (2022)
- [Dark, Beyond Deep: A Paradigm Shift to Cognitive AI with Humanlike Common Sense](https://mlsanity.com/paper/2004.09044/) (2020)
- [Language Grounding with 3D Objects](https://mlsanity.com/paper/2107.12514/) (2021)
- [PartAfford: Part-level Affordance Discovery from 3D Objects](https://mlsanity.com/paper/2202.13519/) (2022)
- [INVIGORATE: Interactive Visual Grounding and Grasping in Clutter](https://mlsanity.com/paper/2108.11092/) (2021)
- [ROI-based Robotic Grasp Detection for Object Overlapping Scenes](https://mlsanity.com/paper/1808.10313/) (2018)
- [Affordances in Robotic Tasks -- A Survey](https://mlsanity.com/paper/2004.07400/) (2020)
- [Same Object, Different Grasps: Data and Semantic Knowledge for Task-Oriented Grasping](https://mlsanity.com/paper/2011.06431/) (2020)
- [Robot Object Retrieval with Contextual Natural Language Queries](https://mlsanity.com/paper/2006.13253/) (2020)

## Built on by (most cited first, 17 of 17)

- [Language Embedded Radiance Fields for Zero-Shot Task-Oriented Grasping](https://mlsanity.com/paper/2309.07970/) (2023)
- [One-Shot Open Affordance Learning with Foundation Models](https://mlsanity.com/paper/2311.17776/) (2023)
- [AffordGrasp: In-Context Affordance Reasoning for Open-Vocabulary Task-Oriented Grasping in Clutter](https://mlsanity.com/paper/2503.00778/) (2025)
- [Scene-Driven Multimodal Knowledge Graph Construction for Embodied AI](https://mlsanity.com/paper/2311.03783/) (2023)
- [GLOVER: Generalizable Open-Vocabulary Affordance Reasoning for Task-Oriented Grasping](https://mlsanity.com/paper/2411.12286/) (2024)
- [GLOVER++: Unleashing the Potential of Affordance Learning from Human Behaviors for Robotic Manipulation](https://mlsanity.com/paper/2505.11865/) (2025)
- [Lightweight Language-driven Grasp Detection using Conditional Consistency Model](https://mlsanity.com/paper/2407.17967/) (2024)
- [Bridging Perspectives: A Survey on Cross-view Collaborative Intelligence with Egocentric-Exocentric Vision](https://mlsanity.com/paper/2506.06253/) (2025)
- [Text2Grasp: Grasp synthesis by text prompts of object grasping parts](https://mlsanity.com/paper/2404.15189/) (2024)
- [Multi-Task Domain Adaptation for Language Grounding with 3D Objects](https://mlsanity.com/paper/2407.02846/) (2024)
- [FunHOI: Annotation-Free 3D Hand-Object Interaction Generation via Functional Text Guidance](https://mlsanity.com/paper/2502.20805/) (2025)
- [OmniDexVLG: Learning Dexterous Grasp Generation from Vision Language Model-Guided Grasp Semantics, Taxonomy and Functional Affordance](https://mlsanity.com/paper/2512.03874/) (2025)
- [Toward Generalist Neural Motion Planners for Robotic Manipulators: Challenges and Opportunities](https://mlsanity.com/paper/2603.24318/) (2026)
- [CompassAD: Intent-Driven 3D Affordance Grounding in Functionally Competing Objects](https://mlsanity.com/paper/2604.02060/) (2026)
- [Hierarchical Transformers for Unsupervised 3D Shape Abstraction](https://mlsanity.com/paper/2510.27088/) (2025)
- [Dynamics-Aware Meta-Imitation for Generalization to Unseen Robotic Manipulation](https://mlsanity.com/paper/2607.15880/) (2026)
- [Zero-shot 2D Grounding with Novel Affordance Types](https://mlsanity.com/paper/2608.08929/) (2026)

## Abstract (arXiv)

Robotic grasping is a fundamental ability for a robot to interact with the environment. Current methods focus on how to obtain a stable and reliable grasping pose in object level, while little work has been studied on part (shape)-wise grasping which is related to fine-grained grasping and robotic affordance. Parts can be seen as atomic elements to compose an object, which contains rich semantic knowledge and a strong correlation with affordance. However, lacking a large part-wise 3D robotic dataset limits the development of part representation learning and downstream applications. In this paper, we propose a new large Language-guided SHape grAsPing datasEt (named LangSHAPE) to promote 3D part-level affordance and grasping ability learning. From the perspective of robotic cognition, we design a two-stage fine-grained robotic grasping framework (named LangPartGPD), including a novel 3D part language grounding model and a part-aware grasp pose detection model, in which explicit language input from human or large language models (LLMs) could guide a robot to generate part-level 6-DoF grasping pose with textual explanation. Our method combines the advantages of human-robot collaboration and LLMs' planning ability using explicit language as a symbolic intermediate. To evaluate the effectiveness of our proposed method, we perform 3D part grounding and fine-grained grasp detection experiments on both simulation and physical robot settings, following language instructions across different degrees of textual complexity. Results show our method achieves competitive performance in 3D geometry fine-grained grounding, object affordance inference, and 3D part-aware grasping tasks. Our dataset and code are available on our project website https://sites.google.com/view/lang-shape