mlsanityData to 7 Oct 2026

Learning Task-Oriented Grasping for Tool Manipulation from Simulated Self-Supervision

arXiv 1806.09266 ↗First posted 25 Jun 2018cs.RO cs.CV cs.LGcited by 50

Not on the map. Why?

Abstract (arXiv)

Tool manipulation is vital for facilitating robots to complete challenging task goals. It requires reasoning about the desired effect of the task and thus properly grasping and manipulating the tool to achieve the task. Task-agnostic grasping optimizes for grasp robustness while ignoring crucial task-specific constraints. In this paper, we propose the Task-Oriented Grasping Network (TOG-Net) to jointly optimize both task-oriented grasping of a tool and the manipulation policy for that tool. The training process of the model is based on large-scale simulated self-supervision with procedurally generated tool objects. We perform both simulated and real-world experiments on two tool-based manipulation tasks: sweeping and hammering. Our model achieves overall 71.1% task success rate for sweeping and 80.0% task success rate for hammering. Supplementary material is available at: bit.ly/task-oriented-grasp

Key facts. Cited by 50 papers in the map (data through 7 Oct 2026). First posted on arXiv 25 Jun 2018.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2021: 16; 2022: 4; 2023: 8; 2024: 5; 2025: 7; 2026: 10.

50 papers.

Sort:

By area:

  1. 2021Contact-GraspNet: Efficient 6-DoF Grasp Generation in Cluttered Scenescited by 213
  2. 2021ManiSkill: Generalizable Manipulation Skill Benchmark with Large-Scale Demonstrationscited by 193
  3. 2022OakInk: A Large-scale Knowledge Repository for Understanding Hand-Object Interactioncited by 129
  4. 2021Synergies Between Affordance and Geometry: 6-DoF Grasp Detection via Implicit Representationscited by 107
  5. 2021ManipulaTHOR: A Framework for Visual Object Manipulationcited by 101
  6. 2021VAT-Mart: Learning Visual Action Trajectory Proposals for Manipulating 3D ARTiculated Objectscited by 78
  7. 2021Semantics for Robotic Mapping, Perception and Interaction: A Surveycited by 56
  8. 2021CaTGrasp: Learning Category-Level Task-Relevant Grasping in Clutter from Simulationcited by 49
  9. 2022DiffSkill: Skill Abstraction from Differentiable Physics for Deformable Object Manipulations with Toolscited by 44
  10. 2023Creative Robot Tool Use with Large Language Modelscited by 42
  11. 2021One-Shot Object Affordance Detection in the Wildcited by 40
  12. 2023M2T2: Multi-Task Masked Transformer for Object-centric Pick and Placecited by 40
38 more
  1. 2024A Survey of Embodied Learning for Object-Centric Robotic Manipulationcited by 34
  2. 2025GraspGen: A Diffusion-based Framework for 6-DOF Grasping with On-Generator Trainingcited by 32
  3. 2022Leveraging Language for Accelerated Learning of Tool Manipulationcited by 26
  4. 2021Learning Visual Affordance Grounding from Demonstration Videoscited by 25
  5. 2021A Joint Network for Grasp Detection Conditioned on Natural Language Commandscited by 23
  6. 2021How to select and use tools? : Active Perception of Target Objects Using Multimodal Deep Learningcited by 15
  7. 2024ShapeGrasp: Zero-Shot Task-Oriented Grasping with Large Language Models through Geometric Decompositioncited by 14
  8. 2023FingerSLAM: Closed-loop Unknown Object Localization and Reconstruction from Visuo-tactile Feedbackcited by 12
  9. 2021Collision-Aware Target-Driven Object Grasping in Constrained Environmentscited by 11
  10. 2025VLMgineer: Vision Language Models as Robotic Toolsmithscited by 9
  11. 2023Learning Reusable Manipulation Strategiescited by 8
  12. 2023Learning to Design and Use Tools for Robotic Manipulationcited by 7
  13. 2021Scene Editing as Teleoperation: A Case Study in 6DoF Kit Assemblycited by 6
  14. 2023Learning Generalizable Tool-use Skills through Trajectory Generationcited by 6
  15. 2024One-Shot Manipulation Strategy Learning by Making Contact Analogiescited by 6
  16. 2021Discovering Generalizable Skills via Automated Generation of Diverse Taskscited by 5
  17. 2024MaxMI: A Maximal Mutual Information Criterion for Manipulation Concept Discoverycited by 5
  18. 2025Task-Oriented 6-DoF Grasp Pose Detection in Clutterscited by 4
  19. 2025RobotSmith: Generative Robotic Tool Design for Acquisition of Complex Manipulation Skillscited by 4
  20. 2024Visual Imitation Learning of Task-Oriented Object Grasping and Rearrangementcited by 3
  21. 2025Afford-X: Generalizable and Slim Affordance Reasoning for Task-oriented Manipulationcited by 3
  22. 2022Fixing Malfunctional Objects With Learned Physical Simulation and Functional Predictioncited by 2
  23. 2023Self-Supervised Learning of Action Affordances as Interaction Modescited by 2
  24. 2023Learning Generalizable Tool Use with Non-rigid Grasp-pose Registrationcited by 2
  25. 2025Point2Act: Efficient 3D Distillation of Multimodal LLMs for Zero-Shot Context-Aware Graspingcited by 2
  26. 2021Text-driven object affordance for guiding grasp-type recognition in multimodal robot teachingcited by 1
  27. 2021Single RGB-D Camera Teleoperation for General Robotic Manipulationcited by 1
  28. 2026Generate, Transfer, Adapt: Learning Functional Dexterous Grasping from a Single Human Demonstrationcited by 1
  29. 2026Imitating What Works: Simulation-Filtered Modular Policy Learning from Human Videoscited by 1
  30. 2025Adaptive Inverse Kinematics Framework for Learning Variable-Length Tool Manipulation in Roboticscited by 0
  31. 2026Creative Robot Tool Use by Counterfactual Reasoningcited by 0
  32. 2026COBALT: Crowdsourcing Robot Learning via Cloud-Based Teleoperation with Smartphonescited by 0
  33. 2026Object Pose and Shape Estimation for Grasping: Does it Work?cited by 0
  34. 2026NDPP-Grasp: Non-Differentiable Physical Plausibility Constraint-Guided Task-Oriented Dexterous Grasp Generationcited by 0
  35. 2026GROW2: Grounding Which and Where for Robot Tool Usecited by 0
  36. 2026FuncBridge: Towards Functional Tool-Use Generalization via Keypoint Trajectory Reasoningcited by 0
  37. 2026Iterative Grasp Pose Refinement: A Deep Reinforcement Learning Approach for 2D Visioncited by 0
  38. 2026EgoTools: Towards Tool-Centric Reasoning in Real-World Egocentric Videoscited by 0