# Task-Oriented Grasp Prediction with Visual-Language Inputs

arXiv: https://arxiv.org/abs/2302.14355 · HTML: https://mlsanity.com/paper/2302.14355/

## Key facts

- Cited by 14 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 28 Feb 2023.

## Built on by (most cited first, 14 of 14)

- [RT-Affordance: Affordances are Versatile Intermediate Representations for Robot Manipulation](https://mlsanity.com/paper/2411.02704/) (2024)
- [Language-guided Robot Grasping: CLIP-based Referring Grasp Synthesis in Clutter](https://mlsanity.com/paper/2311.05779/) (2023)
- [Reasoning Grasping via Multimodal Large Language Model](https://mlsanity.com/paper/2402.06798/) (2024)
- [Language-Driven 6-DoF Grasp Detection Using Negative Prompt Guidance](https://mlsanity.com/paper/2407.13842/) (2024)
- [ShapeGrasp: Zero-Shot Task-Oriented Grasping with Large Language Models through Geometric Decomposition](https://mlsanity.com/paper/2403.18062/) (2024)
- [Text2Grasp: Grasp synthesis by text prompts of object grasping parts](https://mlsanity.com/paper/2404.15189/) (2024)
- [HiFi-CS: Towards Open Vocabulary Visual Grounding For Robotic Grasping Using Vision-Language Models](https://mlsanity.com/paper/2409.10419/) (2024)
- [MoMaGen: Generating Demonstrations under Soft and Hard Constraints for Multi-Step Bimanual Mobile Manipulation](https://mlsanity.com/paper/2510.18316/) (2025)
- [What Breaks Embodied AI Security:LLM Vulnerabilities, CPS Flaws,or Something Else?](https://mlsanity.com/paper/2602.17345/) (2026)
- [Task-Oriented 6-DoF Grasp Pose Detection in Clutters](https://mlsanity.com/paper/2502.16976/) (2025)
- [FunHOI: Annotation-Free 3D Hand-Object Interaction Generation via Functional Text Guidance](https://mlsanity.com/paper/2502.20805/) (2025)
- [Improving Generalization of Language-Conditioned Robot Manipulation](https://mlsanity.com/paper/2508.02405/) (2025)
- [ORACLE-Grasp: Zero-Shot Affordance-Aligned Robotic Grasping using Large Multimodal Models](https://mlsanity.com/paper/2505.08417/) (2025)
- [Grasp-Then-Plan with Failure Attribution: A Closed Two-Stage Framework for Precise and Generalizable Robotic Manipulation](https://mlsanity.com/paper/2606.03385/) (2026)

## Abstract (arXiv)

To perform household tasks, assistive robots receive commands in the form of user language instructions for tool manipulation. The initial stage involves selecting the intended tool (i.e., object grounding) and grasping it in a task-oriented manner (i.e., task grounding). Nevertheless, prior researches on visual-language grasping (VLG) focus on object grounding, while disregarding the fine-grained impact of tasks on object grasping. Task-incompatible grasping of a tool will inevitably limit the success of subsequent manipulation steps. Motivated by this problem, this paper proposes GraspCLIP, which addresses the challenge of task grounding in addition to object grounding to enable task-oriented grasp prediction with visual-language inputs. Evaluation on a custom dataset demonstrates that GraspCLIP achieves superior performance over established baselines with object grounding only. The effectiveness of the proposed method is further validated on an assistive robotic arm platform for grasping previously unseen kitchen tools given the task specification. Our presentation video is available at: https://www.youtube.com/watch?v=e1wfYQPeAXU.