# FoundationGrasp: Generalizable Task-Oriented Grasping with Foundation Models

arXiv: https://arxiv.org/abs/2404.10399 · HTML: https://mlsanity.com/paper/2404.10399/

## Key facts

- Cited by 17 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 16 Apr 2024.

## Built on by (most cited first, 17 of 17)

- [AffordGrasp: In-Context Affordance Reasoning for Open-Vocabulary Task-Oriented Grasping in Clutter](https://mlsanity.com/paper/2503.00778/) (2025)
- [VISO-Grasp: Vision-Language Informed Spatial Object-centric 6-DoF Active View Planning and Grasping in Clutter and Invisibility](https://mlsanity.com/paper/2503.12609/) (2025)
- [Don't Drop the BATON: Long-Horizon Robot Manipulation via Agentic Subtask Exploration and Transition-aware Memory](https://mlsanity.com/paper/2608.16889/) (2026)
- [HGDiffuser: Efficient Task-Oriented Grasp Generation via Human-Guided Grasp Diffusion Models](https://mlsanity.com/paper/2503.00508/) (2025)
- [Point2Act: Efficient 3D Distillation of Multimodal LLMs for Zero-Shot Context-Aware Grasping](https://mlsanity.com/paper/2508.03099/) (2025)
- [OmniDexVLG: Learning Dexterous Grasp Generation from Vision Language Model-Guided Grasp Semantics, Taxonomy and Functional Affordance](https://mlsanity.com/paper/2512.03874/) (2025)
- [Imitating What Works: Simulation-Filtered Modular Policy Learning from Human Videos](https://mlsanity.com/paper/2602.13197/) (2026)
- [GPA-RAM: Grasp-Pretraining Augmented Robotic Attention Mamba for Spatial Task Learning](https://mlsanity.com/paper/2504.19683/) (2025)
- [Tactile-based Reinforcement Learning for Adaptive Grasping under Observation Uncertainties](https://mlsanity.com/paper/2505.16167/) (2025)
- [Adaptive Vision-Language Grasping via Composable Foundation Priors and Generalizable Grasp Synthesis](https://mlsanity.com/paper/2609.04096/) (2026)
- [3D-CDRGP: Towards Cross-Device Robotic Grasping Policy in 3D Open World](https://mlsanity.com/paper/2411.18133/) (2024)
- [ORACLE-Grasp: Zero-Shot Affordance-Aligned Robotic Grasping using Large Multimodal Models](https://mlsanity.com/paper/2505.08417/) (2025)
- [ImaginationPolicy: Towards Generalizable, Precise and Reliable End-to-End Policy for Robotic Manipulation](https://mlsanity.com/paper/2509.20841/) (2025)
- [ConPoSe: LLM-Guided Contact Point Selection for Scalable Cooperative Object Pushing](https://mlsanity.com/paper/2510.08705/) (2025)
- [VLAD-Grasp: Zero-shot Grasp Detection via Vision-Language Models](https://mlsanity.com/paper/2511.05791/) (2025)
- [RoboPCA: Pose-centered Affordance Learning from Human Demonstrations for Robot Manipulation](https://mlsanity.com/paper/2603.07691/) (2026)
- [Grasp-Then-Plan with Failure Attribution: A Closed Two-Stage Framework for Precise and Generalizable Robotic Manipulation](https://mlsanity.com/paper/2606.03385/) (2026)

## Abstract (arXiv)

Task-oriented grasping (TOG), which refers to synthesizing grasps on an object that are configurationally compatible with the downstream manipulation task, is the first milestone towards tool manipulation. Analogous to the activation of two brain regions responsible for semantic and geometric reasoning during cognitive processes, modeling the intricate relationship between objects, tasks, and grasps necessitates rich semantic and geometric prior knowledge about these elements. Existing methods typically restrict the prior knowledge to a closed-set scope, limiting their generalization to novel objects and tasks out of the training set. To address such a limitation, we propose FoundationGrasp, a foundation model-based TOG framework that leverages the open-ended knowledge from foundation models to learn generalizable TOG skills. Extensive experiments are conducted on the contributed Language and Vision Augmented TaskGrasp (LaViA-TaskGrasp) dataset, demonstrating the superiority of FoundationGrasp over existing methods when generalizing to novel object instances, object classes, and tasks out of the training set. Furthermore, the effectiveness of FoundationGrasp is validated in real-robot grasping and manipulation experiments on a 7-DoF robotic arm. Our code, data, appendix, and video are publicly available at https://sites.google.com/view/foundationgrasp.