mlsanityData to 14 Oct 2026

Diffusion Models are Open-World Affordance Learners: Leveraging Generative Priors for 3D Affordance Learning

arXiv 2508.01651 ↗First posted 3 Aug 2025cs.CVcited by 6

On the map: Diffusion models → Video generation

Abstract (arXiv)

3D affordance grounding aims to understand how diverse objects can be manipulated, making it a cornerstone of embodied interaction. However, prior works struggle to generalize to out-of-distribution, open-world scenarios, leaving a critical gap between limited dataset performance and real-world application needs. Inspired by the saying: textit``What I can not create, I do not understand'', we find generative models can generate semantically valid HOI images, which indicates inherent encoding of affordance concepts. Building on this insight, we propose DAG, the first innovative diffusion-based 3D affordance grounding framework that extracts general affordance knowledge from text-to-image diffusion models for 3D affordance prediction. Specifically, we extract the affordance priors from a diffusion model to encode HOI priors, and design an affordance block with a multi-source affordance decoder for dense 3D affordance prediction. Extensive experiments show that DAG consistently outperforms state-of-the-art methods and exhibits strong open-world generalization, even in the challenging one-shot setting. The code of our method is released on textcolorbluetextithttps://github.com/hq-King/DAG.

Key facts. Cited by 6 papers in the map (data through 14 Oct 2026). Placed in Video generation, one of 14 directions in Diffusion models. Builds on 51 of its 75 references that are in the map. First posted on arXiv 3 Aug 2025.

Builds on

51 of 75 references.

Closest ancestors · 2

Same direction, newest first

  1. 2024Beyond the Contact: Discovering Comprehensive Affordance for 3D Objects from Pre-trained 2D Diffusion Models23 Jan
  2. 2023Boosting Human-Object Interaction Detection with Text-to-Image Diffusion Model20 May

Other references · 49

  1. 2021High-Resolution Image Synthesis with Latent Diffusion Modelscited by 18,863
  2. 2021Diffusion Models Beat GANs on Image Synthesiscited by 8,255
  3. 2022Hierarchical Text-Conditional Image Generation with CLIP Latentscited by 6,585
  4. 2022Photorealistic Text-to-Image Diffusion Models with Deep Language Understandingcited by 6,355
  5. 2016V-Net: Fully Convolutional Neural Networks for Volumetric Medical Image Segmentationcited by 2,292
  6. 2021LAION-400M: Open Dataset of CLIP-Filtered 400 Million Image-Text Pairscited by 1,555
  7. 2022eDiff-I: Text-to-Image Diffusion Models with an Ensemble of Expert Denoiserscited by 931
  8. 2021Label-Efficient Semantic Segmentation with Diffusion Modelscited by 499
  9. 2023Emergent Correspondence from Image Diffusioncited by 446
  10. 2023Open-Vocabulary Panoptic Segmentation with Text-to-Image Diffusion Modelscited by 375
  11. 2022DiffusionDet: Diffusion Model for Object Detectioncited by 354
  12. 2023Your Diffusion Model is Secretly a Zero-Shot Classifiercited by 306
37 more
  1. 2023GRES: Generalized Referring Expression Segmentationcited by 254
  2. 2022CrossPoint: Self-Supervised Cross-Modal Contrastive Learning for 3D Point Cloud Understandingcited by 198
  3. 2017PointFusion: Deep Sensor Fusion for 3D Bounding Box Estimationcited by 170
  4. 2023Contrast with Reconstruct: Contrastive 3D Representation Learning Guided by Generative Pretrainingcited by 164
  5. 2023ManipLLM: Embodied Multimodal Large Language Model for Object-Centric Robotic Manipulationcited by 162
  6. 2021Referring Transformer: A One-step Approach to Multi-task Visual Groundingcited by 143
  7. 2025RoboBrain: A Unified Brain Model for Robotic Manipulation from Abstract to Concretecited by 135
  8. 2023Diffuse, Attend, and Segment: Unsupervised Zero-Shot Segmentation using Stable Diffusioncited by 131
  9. 2022Autoencoders as Cross-Modal Teachers: Can Pretrained 2D Image Transformers Help 3D Representation Learning?cited by 122
  10. 2023Affordance Diffusion: Synthesizing Hand-Object Interactionscited by 110
  11. 20213D AffordanceNet: A Benchmark for Visual Object Affordance Understandingcited by 100
  12. 2023Grounding 3D Object Affordance from 2D Interactions in Imagescited by 62
  13. 2023One-Shot Open Affordance Learning with Foundation Modelscited by 45
  14. 2023Open-Vocabulary Affordance Detection in 3D Point Cloudscited by 41
  15. 2023Uni3D: A Unified Baseline for Multi-dataset 3D Object Detectioncited by 39
  16. 2023Point-Cloud Completion with Pretrained Text-to-image Diffusion Modelscited by 37
  17. 2022Cross-modal Learning for Image-Guided Point Cloud Shape Completioncited by 36
  18. 20253D-AffordanceLLM: Harnessing Large Language Models for Open-Vocabulary Affordance Detection in 3D Worldscited by 30
  19. 2024GREAT: Geometry-Intention Collaborative Inference for Open-Vocabulary 3D Object Affordance Groundingcited by 28
  20. 2022Let Images Give You More:Point Cloud Cross-Modal Training for Shape Analysiscited by 24
  21. 2024Learning 2D Invariant Affordance Knowledge for 3D Affordance Groundingcited by 22
  22. 2025AffordGrasp: In-Context Affordance Reasoning for Open-Vocabulary Task-Oriented Grasping in Cluttercited by 20
  23. 2025Grounding 3D Object Affordance with Language Instructions, Visual Observations and Interactionscited by 19
  24. 2024Where's Waldo: Diffusion Features for Personalized Segmentation and Retrievalcited by 18
  25. 2024SeqAfford: Sequential 3D Affordance Reasoning via Multimodal Large Language Modelcited by 18
  26. 2025OpenHOI: Open-World Hand-Object Interaction Synthesis with Multimodal Large Language Modelcited by 17
  27. 2023PointDC:Unsupervised Semantic Segmentation of 3D Point Clouds via Cross-modal Distillation and Super-Voxel Clusteringcited by 15
  28. 2023PointMBF: A Multi-scale Bidirectional Fusion Network for Unsupervised RGB-D Point Cloud Registrationcited by 14
  29. 2024Neural Residual Diffusion Models for Deep Scalable Vision Generationcited by 9
  30. 2024Explicitly Guided Information Interaction Network for Cross-modal Point Cloud Completioncited by 9
  31. 2025GenPC: Zero-shot Point Cloud Completion via 3D Generative Priorscited by 9
  32. 2025Weakly-Supervised Affordance Grounding Guided by Part-Level Semantic Priorscited by 9
  33. 2023LMD: Faster Image Reconstruction with Latent Masking Diffusioncited by 7
  34. 2022PointCMC: Cross-Modal Multi-Scale Correspondences Learning for Point Cloud Understandingcited by 5
  35. 2023Fine-grained Text and Image Guided Point Cloud Completion with CLIP Modelcited by 5
  36. 2024DMF-Net: Image-Guided Point Cloud Completion with Dual-Channel Modality Fusion and Shape-Aware Upsampling Transformercited by 5
  37. 2024CMR-Agent: Learning a Cross-Modal Agent for Iterative Image-to-Point Cloud Registrationcited by 3

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2025: 2; 2026: 4.

6 papers.

Similar, not cited

Close in wording, no citation either way

  1. 2024INTRA: Interaction Relationship-aware Weakly Supervised Affordance Grounding
  2. 2026AffordAny: Open-World 3D Affordance Grounding from Monocular RGB Images via Vision-Language-Guided Geometric Reasoning
  3. 2025DAViD: Modeling Dynamic Affordance of 3D Objects Using Pre-trained Video Diffusion Models
  4. 2024AffordanceLLM: Grounding Affordance from Vision Language Models
  5. 2024Move as You Say, Interact as You Can: Language-guided Human Motion Generation with Scene Affordance
  6. 2026HAMMER: Harnessing MLLM via Cross-Modal Integration for Intention-Driven 3D Affordance Grounding
  7. 2026ThinkAfford: Affordance-Centric Reasoning for Fine-Grained 3D Grounding in Cluttered Scenes
  8. 2026VAGNet: Grounding 3D Affordance from Human-Object Interactions in Videos