mlsanityData to 14 Oct 2026

Affogato: Open-Vocabulary Affordance Grounding with Automated Data Generation at Scale

arXiv 2506.12009 ↗First posted 13 Jun 2025cs.CVcited by 6

Not on the map. Why?

Abstract (arXiv)

Affordance grounding aims to localize where to interact with an object, a fundamental capability for embodied agents. Yet progress is bottlenecked by data: manual annotation is prohibitively expensive and confines existing datasets to a narrow set of predefined object and affordance categories. We introduce Affogato, a framework for open-vocabulary affordance grounding centered on Affogato-750K, a large-scale dataset of 750K 3D affordance heatmaps paired with natural language queries. We build it with a fully automated pipeline that orchestrates foundation models to generate them at scale without human labeling. It covers significantly more diverse categories than any existing dataset. For reliable evaluation, we further provide 5K human-verified test pairs. We also present Espresso-3D and Espresso-2D, simple yet effective models with a unified architecture across both modalities. Pretraining on Affogato-750K improves both Espresso and prior methods and yields the largest gains on unseen object and affordance categories, showing that it provides broadly transferable supervision across architectures.

Key facts. Cited by 6 papers in the map (data through 14 Oct 2026). Builds on 59 of its 78 references that are in the map. First posted on arXiv 13 Jun 2025.

Builds on

59 of 78 references.

References · 59

  1. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,744
  2. 2022Chain-of-Thought Prompting Elicits Reasoning in Large Language Modelscited by 12,459
  3. 2023DINOv2: Learning Robust Visual Features without Supervisioncited by 7,974
  4. 2024SAM 2: Segment Anything in Images and Videoscited by 2,863
  5. 2021Masked-attention Mask Transformer for Universal Image Segmentationcited by 2,351
  6. 2016V-Net: Fully Convolutional Neural Networks for Volumetric Medical Image Segmentationcited by 2,292
  7. 2025Gemma 3 Technical Reportcited by 1,824
  8. 2022Objaverse: A Universe of Annotated 3D Objectscited by 1,540
  9. 2021Per-Pixel Classification is Not All You Need for Semantic Segmentationcited by 1,152
  10. 2016Modeling Context in Referring Expressionscited by 1,063
  11. 2023Open X-Embodiment: Robotic Learning Datasets and RT-X Modelscited by 862
  12. 2023LISA: Reasoning Segmentation via Large Language Modelcited by 776
38 more
  1. 2018PartNet: A Large-scale Benchmark for Fine-grained and Hierarchical Part-level 3D Object Understandingcited by 526
  2. 2023Faster Segment Anything: Towards Lightweight SAM for Mobile Applicationscited by 352
  3. 2019Point-Voxel CNN for Efficient 3D Deep Learningcited by 343
  4. 2023PointLLM: Empowering Large Language Models to Understand Point Cloudscited by 333
  5. 2024Molmo and PixMo: Open Weights and Open Data for State-of-the-Art Vision-Language Modelscited by 329
  6. 2022OneFormer: One Transformer to Rule Universal Image Segmentationcited by 328
  7. 2022Mask3D: Mask Transformer for 3D Semantic Instance Segmentationcited by 303
  8. 2023GRES: Generalized Referring Expression Segmentationcited by 254
  9. 2023Affordances from Human Videos as a Versatile Representation for Roboticscited by 226
  10. 2023ULIP-2: Towards Scalable Multimodal Pre-training for 3D Understandingcited by 202
  11. 2016What do different evaluation metrics tell us about saliency models?cited by 185
  12. 2021Referring Transformer: A One-step Approach to Multi-task Visual Groundingcited by 143
  13. 2018Grounded Human-Object Interaction Hotspots from Videocited by 129
  14. 2022Joint Hand Motion and Interaction Hotspots Prediction from Egocentric Videoscited by 111
  15. 20223D-SPS: Single-Stage 3D Visual Grounding via Referred Point Progressive Selectioncited by 110
  16. 20213D AffordanceNet: A Benchmark for Visual Object Affordance Understandingcited by 100
  17. 2024SAMPart3D: Segment Any Part in 3D Objectscited by 97
  18. 2023OneFormer3D: One Transformer for Unified Point Cloud Segmentationcited by 95
  19. 2025PARTFIELD: Learning 3D Feature Fields for Part Segmentation and Beyondcited by 95
  20. 2023RegionPLC: Regional Point-Language Contrastive Learning for Open-World 3D Scene Understandingcited by 94
  21. 2024What If We Recaption Billions of Web Images with LLaMA-3?cited by 87
  22. 2022Learning Affordance Grounding from Exocentric Imagescited by 82
  23. 2017Learning to Act Properly: Predicting and Explaining Affordances from Imagescited by 74
  24. 2024Robo-ABC: Affordance Generalization Beyond Categories via Semantic Correspondence for Robot Manipulationcited by 71
  25. 2023LOCATE: Localize and Transfer Object Parts for Weakly Supervised Affordance Groundingcited by 64
  26. 2023Grounding 3D Object Affordance from 2D Interactions in Imagescited by 62
  27. 2024AffordanceLLM: Grounding Affordance from Vision Language Modelscited by 55
  28. 2023One-Shot Open Affordance Learning with Foundation Modelscited by 45
  29. 2023Open-Vocabulary Affordance Detection in 3D Point Cloudscited by 41
  30. 2024In Defense of Lazy Visual Grounding for Open-Vocabulary Semantic Segmentationcited by 38
  31. 2022PartAfford: Part-level Affordance Discovery from 3D Objectscited by 36
  32. 2023Understanding 3D Object Interaction from a Single Imagecited by 33
  33. 2024Beyond the Contact: Discovering Comprehensive Affordance for 3D Objects from Pre-trained 2D Diffusion Modelscited by 32
  34. 2022One-Shot Transfer of Affordance Regions? AffCorrs!cited by 31
  35. 20253D-AffordanceLLM: Harnessing Large Language Models for Open-Vocabulary Affordance Detection in 3D Worldscited by 30
  36. 2020Affordances in Robotic Tasks -- A Surveycited by 28
  37. 2025MMR: A Large-scale Benchmark Dataset for Multi-target and Multi-granularity Reasoning Segmentationcited by 28
  38. 2022Grounded Affordance from Exocentric Viewcited by 22

50 most cited of 59 All 59 (JSON)

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2026: 6.

6 papers.

  1. 2026AffordVLA: Injecting Affordance Representations into Vision-Language-Action Models via Implicit Feature Alignmentcited by 5
  2. 2026VideoAfford: Grounding 3D Affordance from Human-Object-Interaction Videos via Multimodal Large Language Modelcited by 2
  3. 2026Affostruction: 3D Affordance Grounding with Generative Reconstructioncited by 1
  4. 2026CompassAD: Intent-Driven 3D Affordance Grounding in Functionally Competing Objectscited by 1
  5. 2026Probing and Bridging Geometry-Interaction Cues for Affordance Reasoning in Vision Foundation Modelscited by 0
  6. 2026SceneParser: Hierarchical Scene Parsing for Visual Semantics Understandingcited by 0