# GraspMolmo: Generalizable Task-Oriented Grasping via Large-Scale Synthetic Data Generation

arXiv: https://arxiv.org/abs/2505.13441 · HTML: https://mlsanity.com/paper/2505.13441/

## Key facts

- Cited by 9 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 19 May 2025.

## Built on by (most cited first, 9 of 9)

- [GraspGen: A Diffusion-based Framework for 6-DOF Grasping with On-Generator Training](https://mlsanity.com/paper/2507.13097/) (2025)
- [AffordanceVLA: A Vision-Language-Action Model Empowering Action Generation through Affordance-Aware Understanding](https://mlsanity.com/paper/2606.06155/) (2026)
- [Embodied-R1.5: Evolving Physical Intelligence via Embodied Foundation Models](https://mlsanity.com/paper/2606.11324/) (2026)
- [BOP-ASK: Object-Interaction Reasoning for Vision-Language Models](https://mlsanity.com/paper/2511.16857/) (2025)
- [Poivre: Self-Refining Visual Pointing with Reinforcement Learning](https://mlsanity.com/paper/2509.23746/) (2025)
- [Why Far Looks Up: Probing Spatial Representation in Vision-Language Models](https://mlsanity.com/paper/2605.30161/) (2026)
- [Point2Act: Efficient 3D Distillation of Multimodal LLMs for Zero-Shot Context-Aware Grasping](https://mlsanity.com/paper/2508.03099/) (2025)
- [VCoT-Grasp: Grasp Foundation Models with Visual Chain-of-Thought Reasoning for Language-driven Grasp Generation](https://mlsanity.com/paper/2510.05827/) (2025)
- [Beyond the Current Scene: Event-Referential Grasping with Active View Selection](https://mlsanity.com/paper/2609.39375/) (2026)

## Abstract (arXiv)

We present GrasMolmo, a generalizable open-vocabulary task-oriented grasping (TOG) model. GraspMolmo predicts semantically appropriate, stable grasps conditioned on a natural language instruction and a single RGB-D frame. For instance, given "pour me some tea", GraspMolmo selects a grasp on a teapot handle rather than its body. Unlike prior TOG methods, which are limited by small datasets, simplistic language, and uncluttered scenes, GraspMolmo learns from PRISM, a novel large-scale synthetic dataset of 379k samples featuring cluttered environments and diverse, realistic task descriptions. We fine-tune the Molmo visual-language model on this data, enabling GraspMolmo to generalize to novel open-vocabulary instructions and objects. In challenging real-world evaluations, GraspMolmo achieves state-of-the-art results, with a 70% prediction success on complex tasks, compared to the 35% achieved by the next best alternative. GraspMolmo also successfully demonstrates the ability to predict semantically correct bimanual grasps zero-shot. We release our synthetic dataset, code, model, and benchmarks to accelerate research in task-semantic robotic manipulation, which, along with videos, are available at https://abhaybd.github.io/GraspMolmo/.