mlsanityData to 7 Oct 2026

GraspGPT: Leveraging Semantic Knowledge from a Large Language Model for Task-Oriented Grasping

arXiv 2307.13204 ↗First posted 25 Jul 2023cs.ROcited by 40

Not on the map. Why?

Abstract (arXiv)

Task-oriented grasping (TOG) refers to the problem of predicting grasps on an object that enable subsequent manipulation tasks. To model the complex relationships between objects, tasks, and grasps, existing methods incorporate semantic knowledge as priors into TOG pipelines. However, the existing semantic knowledge is typically constructed based on closed-world concept sets, restraining the generalization to novel concepts out of the pre-defined sets. To address this issue, we propose GraspGPT, a large language model (LLM) based TOG framework that leverages the open-end semantic knowledge from an LLM to achieve zero-shot generalization to novel concepts. We conduct experiments on Language Augmented TaskGrasp (LA-TaskGrasp) dataset and demonstrate that GraspGPT outperforms existing TOG methods on different held-out settings when generalizing to novel concepts out of the training set. The effectiveness of GraspGPT is further validated in real-robot experiments. Our code, data, appendix, and video are publicly available at https://sites.google.com/view/graspgpt/.

Key facts. Cited by 40 papers in the map (data through 7 Oct 2026). First posted on arXiv 25 Jul 2023.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2023: 2; 2024: 15; 2025: 19; 2026: 4.

40 papers.

Sort:

By area:

  1. 2024RoboSpatial: Teaching Spatial Understanding to 2D and 3D Vision-Language Models for Roboticscited by 133
  2. 2025GraspVLA: a Grasping Foundation Model Pre-trained on Billion-scale Synthetic Action Datacited by 120
  3. 2025GraspGen: A Diffusion-based Framework for 6-DOF Grasping with On-Generator Trainingcited by 32
  4. 2023An In-depth Survey of Large Language Model-based Artificial Intelligence Agentscited by 30
  5. 2024SemGrasp: Semantic Grasp Generation via Language Aligned Discretizationcited by 29
  6. 2023Kinematic-aware Prompting for Generalizable Articulated Object Manipulation with LLMscited by 19
  7. 2024Probabilistically Correct Language-based Multi-Robot Planning using Conformal Predictioncited by 18
  8. 2024KiGRAS: Kinematic-Driven Generative Model for Realistic Agent Simulationcited by 17
  9. 2024Language-Driven 6-DoF Grasp Detection Using Negative Prompt Guidancecited by 16
  10. 2024ShapeGrasp: Zero-Shot Task-Oriented Grasping with Large Language Models through Geometric Decompositioncited by 14
  11. 2024Traffic Scene Generation from Natural Language Description for Autonomous Vehicles with Large Language Modelcited by 14
  12. 2024Towards Open-World Grasping with Large Vision-Language Modelscited by 13
28 more
  1. 2024NL2Contact: Natural Language Guided 3D Hand-Object Contact Modeling with Diffusion Modelcited by 10
  2. 2025FUNCTO: Function-Centric One-Shot Imitation Learning for Tool Manipulationcited by 8
  3. 2025TOUCH: Text-guided Controllable Generation of Free-Form Hand-Object Interactionscited by 8
  4. 2024Text2Grasp: Grasp synthesis by text prompts of object grasping partscited by 7
  5. 2025MoMaGen: Generating Demonstrations under Soft and Hard Constraints for Multi-Step Bimanual Mobile Manipulationcited by 7
  6. 2025G-DexGrasp: Generalizable Dexterous Grasping Synthesis Via Part-Aware Prior Retrieval and Prior-Assisted Generationcited by 5
  7. 2025Physically Ground Commonsense Knowledge for Articulated Object Manipulation with Analytic Conceptscited by 5
  8. 2024Robot Instance Segmentation with Few Annotations for Graspingcited by 4
  9. 2025Enhancing Large Vision Model in Street Scene Semantic Understanding through Leveraging Posterior Optimization Trajectorycited by 4
  10. 2025Task-Oriented 6-DoF Grasp Pose Detection in Clutterscited by 4
  11. 2025PosA-VLA: Enhancing Action Generation via Pose-Conditioned Anchor Attentioncited by 4
  12. 2024GraspMamba: A Mamba-based Language-driven Grasp Detection Framework with Hierarchical Feature Learningcited by 3
  13. 2025HGDiffuser: Efficient Task-Oriented Grasp Generation via Human-Guided Grasp Diffusion Modelscited by 3
  14. 2024Reflectance Estimation for Proximity Sensing by Vision-Language Models: Utilizing Distributional Semantics for Low-Level Cognition in Roboticscited by 2
  15. 2025GraspCoT: Integrating Physical Property Reasoning for 6-DoF Grasping under Flexible Language Instructionscited by 2
  16. 2025Point2Act: Efficient 3D Distillation of Multimodal LLMs for Zero-Shot Context-Aware Graspingcited by 2
  17. 2025VCoT-Grasp: Grasp Foundation Models with Visual Chain-of-Thought Reasoning for Language-driven Grasp Generationcited by 2
  18. 2025OmniDexVLG: Learning Dexterous Grasp Generation from Vision Language Model-Guided Grasp Semantics, Taxonomy and Functional Affordancecited by 2
  19. 2024Transparent Object Depth Completioncited by 1
  20. 2024Semantic-Geometric-Physical-Driven Robot Manipulation Skill Transfer via Skill Library and Tactile Representationcited by 1
  21. 2025Label Anything: An Interpretable, High-Fidelity and Prompt-Free Annotatorcited by 1
  22. 2025CLASP: General-Purpose Clothes Manipulation with Semantic Keypointscited by 1
  23. 2026Imitating What Works: Simulation-Filtered Modular Policy Learning from Human Videoscited by 1
  24. 2026RealVLG-R1: A Large-Scale Real-World Visual-Language Grounding Benchmark for Robotic Perception and Manipulationcited by 1
  25. 2026Adaptive Vision-Language Grasping via Composable Foundation Priors and Generalizable Grasp Synthesiscited by 1
  26. 2025ORACLE-Grasp: Zero-Shot Affordance-Aligned Robotic Grasping using Large Multimodal Modelscited by 0
  27. 2025VLAD-Grasp: Zero-shot Grasp Detection via Vision-Language Modelscited by 0
  28. 2026GraspLDP: Towards Generalizable Grasping Policy via Latent Diffusioncited by 0