QwenGrasp: A Usage of Large Vision-Language Model for Target-Oriented Grasping
On the map: Multimodal LLMs / vision-language models → Embodied and 3D vision-language models
Abstract (arXiv)
Target-oriented grasping in unstructured scenes with language control is essential for intelligent robot arm grasping. The ability for the robot arm to understand the human language and execute corresponding grasping actions is a pivotal challenge. In this paper, we propose a combination model called QwenGrasp which combines a large vision-language model with a 6-DoF grasp neural network. QwenGrasp is able to conduct a 6-DoF grasping task on the target object with textual language instruction. We design a complete experiment with six-dimension instructions to test the QwenGrasp when facing with different cases. The results show that QwenGrasp has a superior ability to comprehend the human intention. Even in the face of vague instructions with descriptive words or instructions with direction information, the target object can be grasped accurately. When QwenGrasp accepts the instruction which is not feasible or not relevant to the grasping task, our approach has the ability to suspend the task execution and provide a proper feedback to humans, improving the safety. In conclusion, with the great power of large vision-language model, QwenGrasp can be applied in the open language environment to conduct the target-oriented grasping task with freely input instructions.
Key facts. No citing papers in the map yet (data through 14 Oct 2026). Placed in Embodied and 3D vision-language models, one of 16 directions in Multimodal LLMs / vision-language models. Builds on 17 of its 22 references that are in the map. First posted on arXiv 28 Sep 2023.
Builds on
17 of 22 references.
Closest ancestors · 1
Same direction, newest first
Other references · 16
- 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,744
- 2020Language Models are Few-Shot Learnerscited by 32,419
- 2023Llama 2: Open Foundation and Fine-Tuned Chat Modelscited by 13,028
- 2022PaLM: Scaling Language Modeling with Pathwayscited by 5,326
- 2016PointNet: Deep Learning on Point Sets for 3D Classification and Segmentationcited by 5,248
- 2022Do As I Can, Not As I Say: Grounding Language in Robotic Affordancescited by 2,348
- 2023VoxPoser: Composable 3D Value Maps for Robotic Manipulation with Language Modelscited by 750
- 20196-DOF GraspNet: Variational Grasp Generation for Object Manipulationcited by 242
- 2017Robotic Pick-and-Place of Novel Objects in Clutter with Multi-Affordance Grasping and Cross-Domain Image Matchingcited by 125
- 2018PointNetGPD: Detecting Grasp Configurations from Point Setscited by 80
- 20196-DOF Grasping for Target-driven Object Manipulation in Cluttercited by 70
- 2019S4G: Amodal Single-view Single-Shot SE(3) Grasp Detection in Cluttered Scenescited by 51
4 more
- 2017Multi-Task Domain Adaptation for Deep Learning of Instance Grasping from Simulationcited by 36
- 2023A Joint Modeling of Vision-Language-Action for Target-oriented Grasping in Cluttercited by 34
- 2022MonoGraspNet: 6-DoF Grasping with a Single RGB Imagecited by 24
- 2021GATER: Learning Grasp-Action-Target Embeddings and Relations for Task-Specific Graspingcited by 4
Built on by
Not cited yet.
Similar, not cited
Close in wording, no citation either way
- 2024Reasoning Grasping via Multimodal Large Language Model
- 2025MapleGrasp: Mask-guided Feature Pooling for Language-driven Efficient Robotic Grasping
- 2026SeededGrasp: Language-Guided Grasping in Complex Scenes with Multiple Embodiments
- 2024A Parameter-Efficient Tuning Framework for Language-guided Object Grounding and Robot Grasping
- 2024Language-Driven 6-DoF Grasp Detection Using Negative Prompt Guidance
- 2025GraspCoT: Integrating Physical Property Reasoning for 6-DoF Grasping under Flexible Language Instructions
- 2024Task-Aware Robotic Grasping by evaluating Quality Diversity Solutions through Foundation Models
- 2025Free-form language-based robotic reasoning and grasping