# OVAL-Prompt: Open-Vocabulary Affordance Localization for Robot Manipulation through LLM Affordance-Grounding

arXiv: https://arxiv.org/abs/2404.11000 · HTML: https://mlsanity.com/paper/2404.11000/

## Key facts

- Cited by 15 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 17 Apr 2024.

## Built on by (most cited first, 15 of 15)

- [Learning Precise Affordances from Egocentric Videos for Robotic Manipulation](https://mlsanity.com/paper/2408.10123/) (2024)
- [Affordance-R1: Reinforcement Learning for Generalizable Affordance Reasoning in Multimodal Large Language Model](https://mlsanity.com/paper/2508.06206/) (2025)
- [AffordGrasp: In-Context Affordance Reasoning for Open-Vocabulary Task-Oriented Grasping in Clutter](https://mlsanity.com/paper/2503.00778/) (2025)
- [AffordBot: 3D Fine-grained Embodied Reasoning via Multimodal Large Language Models](https://mlsanity.com/paper/2511.10017/) (2025)
- [Weakly-Supervised Affordance Grounding Guided by Part-Level Semantic Priors](https://mlsanity.com/paper/2505.24103/) (2025)
- [Robot Manipulation in Salient Vision through Referring Image Segmentation and Geometric Constraints](https://mlsanity.com/paper/2409.11518/) (2024)
- [Affordance Agent Harness: Verification-Gated Skill Orchestration](https://mlsanity.com/paper/2605.00663/) (2026)
- [Resource-Efficient Affordance Grounding with Complementary Depth and Semantic Prompts](https://mlsanity.com/paper/2503.02600/) (2025)
- [O3Afford: One-Shot 3D Object-to-Object Affordance Grounding for Generalizable Robotic Manipulation](https://mlsanity.com/paper/2509.06233/) (2025)
- [Visual Affordance Prediction: Survey and Reproducibility](https://mlsanity.com/paper/2505.05074/) (2025)
- [UniDiffGrasp: A Unified Framework Integrating VLM Reasoning and VLM-Guided Part Diffusion for Open-Vocabulary Constrained Grasping with Dual Arms](https://mlsanity.com/paper/2505.06832/) (2025)
- [General-Purpose Robotic Navigation via LVLM-Orchestrated Perception, Reasoning, and Acting](https://mlsanity.com/paper/2506.17462/) (2025)
- [Agentic Scene Policies: Unifying Space, Semantics, and Affordances for Robot Action](https://mlsanity.com/paper/2509.19571/) (2025)
- [Probing and Bridging Geometry-Interaction Cues for Affordance Reasoning in Vision Foundation Models](https://mlsanity.com/paper/2602.20501/) (2026)
- [Zero-shot 2D Grounding with Novel Affordance Types](https://mlsanity.com/paper/2608.08929/) (2026)

## Abstract (arXiv)

In order for robots to interact with objects effectively, they must understand the form and function of each object they encounter. Essentially, robots need to understand which actions each object affords, and where those affordances can be acted on. Robots are ultimately expected to operate in unstructured human environments, where the set of objects and affordances is not known to the robot before deployment (i.e. the open-vocabulary setting). In this work, we introduce OVAL-Prompt, a prompt-based approach for open-vocabulary affordance localization in RGB-D images. By leveraging a Vision Language Model (VLM) for open-vocabulary object part segmentation and a Large Language Model (LLM) to ground each part-segment-affordance, OVAL-Prompt demonstrates generalizability to novel object instances, categories, and affordances without domain-specific finetuning. Quantitative experiments demonstrate that without any finetuning, OVAL-Prompt achieves localization accuracy that is competitive with supervised baseline models. Moreover, qualitative experiments show that OVAL-Prompt enables affordance-based robot manipulation of open-vocabulary object instances and categories. Project Page: https://ekjt.github.io/OVAL-Prompt/