# Task-Oriented 6-DoF Grasp Pose Detection in Clutters

arXiv: https://arxiv.org/abs/2502.16976 · HTML: https://mlsanity.com/paper/2502.16976/

## Key facts

- Cited by 4 papers in the map (data through 14 Oct 2026).
- Builds on 41 of its 54 references that are in the map.
- First posted on arXiv 24 Feb 2025.

## Builds on (most cited first, 30 of 41)

- [PointNet++: Deep Hierarchical Feature Learning on Point Sets in a Metric Space](https://mlsanity.com/paper/1706.02413/) (2017)
- [Learning Hand-Eye Coordination for Robotic Grasping with Deep Learning and Large-Scale Data Collection](https://mlsanity.com/paper/1603.02199/) (2016)
- [6-DOF GraspNet: Variational Grasp Generation for Object Manipulation](https://mlsanity.com/paper/1905.10520/) (2019)
- [Contact-GraspNet: Efficient 6-DoF Grasp Generation in Cluttered Scenes](https://mlsanity.com/paper/2103.14127/) (2021)
- [ACRONYM: A Large-Scale Grasp Dataset Based on Simulation](https://mlsanity.com/paper/2011.09584/) (2020)
- [Grasp Pose Detection in Point Clouds](https://mlsanity.com/paper/1706.09911/) (2017)
- [DocPedia: Unleashing the Power of Large Multimodal Model in the Frequency Domain for Versatile Document Understanding](https://mlsanity.com/paper/2311.11810/) (2023)
- [3D AffordanceNet: A Benchmark for Visual Object Affordance Understanding](https://mlsanity.com/paper/2103.16397/) (2021)
- [MTVQA: Benchmarking Multilingual Text-Centric Visual Question Answering](https://mlsanity.com/paper/2405.11985/) (2024)
- [PointNetGPD: Detecting Grasp Configurations from Point Sets](https://mlsanity.com/paper/1809.06267/) (2018)
- [TabPedia: Towards Comprehensive Visual Table Understanding with Concept Synergy](https://mlsanity.com/paper/2406.01326/) (2024)
- [Few Could Be Better Than All: Feature Sampling and Grouping for Scene Text Detection](https://mlsanity.com/paper/2203.15221/) (2022)
- [TextSquare: Scaling up Text-Centric Visual Instruction Tuning](https://mlsanity.com/paper/2404.12803/) (2024)
- [A Bounding Box is Worth One Token: Interleaving Layout and Text in a Large Language Model for Document Understanding](https://mlsanity.com/paper/2407.01976/) (2024)
- [Harmonizing Visual Text Comprehension and Generation](https://mlsanity.com/paper/2407.16364/) (2024)
- [Learning Task-Oriented Grasping for Tool Manipulation from Simulated Self-Supervision](https://mlsanity.com/paper/1806.09266/) (2018)
- [SpeedFolding: Learning Efficient Bimanual Folding of Garments](https://mlsanity.com/paper/2208.10552/) (2022)
- [Graspness Discovery in Clutters for Fast and Accurate Grasp Detection](https://mlsanity.com/paper/2406.11142/) (2024)
- [GraspGPT: Leveraging Semantic Knowledge from a Large Language Model for Task-Oriented Grasping](https://mlsanity.com/paper/2307.13204/) (2023)
- [ParGo: Bridging Vision-Language with Partial and Global Views](https://mlsanity.com/paper/2408.12928/) (2024)
- [PointNet++ Grasping: Learning An End-to-end Spatial Grasp Generation Algorithm from Sparse Point Clouds](https://mlsanity.com/paper/2003.09644/) (2020)
- [EgoExo-Fitness: Towards Egocentric and Exocentric Full-Body Action Understanding](https://mlsanity.com/paper/2406.08877/) (2024)
- [Same Object, Different Grasps: Data and Semantic Knowledge for Task-Oriented Grasping](https://mlsanity.com/paper/2011.06431/) (2020)
- [MCTBench: Multimodal Cognition towards Text-Rich Visual Scenes Benchmark](https://mlsanity.com/paper/2410.11538/) (2024)
- [Reactive Human-to-Robot Handovers of Arbitrary Objects](https://mlsanity.com/paper/2011.08961/) (2020)
- [An Economic Framework for 6-DoF Grasp Detection](https://mlsanity.com/paper/2407.08366/) (2024)
- [Event-Guided Procedure Planning from Instructional Videos with Text Supervision](https://mlsanity.com/paper/2308.08885/) (2023)
- [Diversifying Spatial-Temporal Perception for Video Domain Generalization](https://mlsanity.com/paper/2310.17942/) (2023)
- [Visuomotor Mechanical Search: Learning to Retrieve Target Objects in Clutter](https://mlsanity.com/paper/2008.06073/) (2020)
- [Dynamic Grasping with Reachability and Motion Awareness](https://mlsanity.com/paper/2103.10562/) (2021)

## Built on by (most cited first, 4 of 4)

- [Rethinking Bimanual Robotic Manipulation: Learning with Decoupled Interaction Framework](https://mlsanity.com/paper/2503.09186/) (2025)
- [Modeling Multiple Normal Action Representations for Error Detection in Procedural Tasks](https://mlsanity.com/paper/2503.22405/) (2025)
- [VLANeXt: Recipes for Building Strong VLA Models](https://mlsanity.com/paper/2602.18532/) (2026)
- [A Closed-Loop Multi-Agent Framework for Robust Multi-Robot Manipulation](https://mlsanity.com/paper/2607.06990/) (2026)

## Abstract (arXiv)

In general, humans would grasp an object differently for different tasks, e.g., "grasping the handle of a knife to cut" vs. "grasping the blade to hand over". In the field of robotic grasp pose detection research, some existing works consider this task-oriented grasping and made some progress, but they are generally constrained by low-DoF gripper type or non-cluttered setting, which is not applicable for human assistance in real life. With an aim to get more general and practical grasp models, in this paper, we investigate the problem named Task-Oriented 6-DoF Grasp Pose Detection in Clutters (TO6DGC), which extends the task-oriented problem to a more general 6-DOF Grasp Pose Detection in Cluttered (multi-object) scenario. To this end, we construct a large-scale 6-DoF task-oriented grasping dataset, 6-DoF Task Grasp (6DTG), which features 4391 cluttered scenes with over 2 million 6-DoF grasp poses. Each grasp is annotated with a specific task, involving 6 tasks and 198 objects in total. Moreover, we propose One-Stage TaskGrasp (OSTG), a strong baseline to address the TO6DGC problem. Our OSTG adopts a task-oriented point selection strategy to detect where to grasp, and a task-oriented grasp generation module to decide how to grasp given a specific task. To evaluate the effectiveness of OSTG, extensive experiments are conducted on 6DTG. The results show that our method outperforms various baselines on multiple metrics. Real robot experiments also verify that our OSTG has a better perception of the task-oriented grasp points and 6-DoF grasp poses.