mlsanityData to 7 Oct 2026

Grounding 3D Object Affordance from 2D Interactions in Images

arXiv 2303.10437 ↗First posted 18 Mar 2023cs.CVcited by 62

Not on the map. Why?

Abstract (arXiv)

Grounding 3D object affordance seeks to locate objects' ''action possibilities'' regions in the 3D space, which serves as a link between perception and operation for embodied agents. Existing studies primarily focus on connecting visual affordances with geometry structures, e.g. relying on annotations to declare interactive regions of interest on the object and establishing a mapping between the regions and affordances. However, the essence of learning object affordance is to understand how to use it, and the manner that detaches interactions is limited in generalization. Normally, humans possess the ability to perceive object affordances in the physical world through demonstration images or videos. Motivated by this, we introduce a novel task setting: grounding 3D object affordance from 2D interactions in images, which faces the challenge of anticipating affordance through interactions of different sources. To address this problem, we devise a novel Interaction-driven 3D Affordance Grounding Network (IAG), which aligns the region feature of objects from different sources and models the interactive contexts for 3D object affordance grounding. Besides, we collect a Point-Image Affordance Dataset (PIAD) to support the proposed task. Comprehensive experiments on PIAD demonstrate the reliability of the proposed task and the superiority of our method. The project is available at https://github.com/yyvhang/IAGNet.

Key facts. Cited by 62 papers in the map (data through 7 Oct 2026). Builds on 54 of its 101 references that are in the map. First posted on arXiv 18 Mar 2023.

Builds on

54 of 101 references.

References · 54

  1. 2015Deep Residual Learning for Image Recognitioncited by 51,809
  2. 2017Mask R-CNNcited by 7,259
  3. 2016Grad-CAM: Visual Explanations from Deep Networks via Gradient-based Localizationcited by 6,690
  4. 2016Feature Pyramid Networks for Object Detectioncited by 5,021
  5. 2017PointNet++: Deep Hierarchical Feature Learning on Point Sets in a Metric Spacecited by 4,384
  6. 2016V-Net: Fully Convolutional Neural Networks for Volumetric Medical Image Segmentationcited by 2,284
  7. 2019Habitat: A Platform for Embodied AI Researchcited by 1,163
  8. 2017Multimodal Machine Learning: A Survey and Taxonomycited by 1,004
  9. 2017Frustum PointNets for 3D Object Detection from RGB-D Datacited by 640
  10. 2018PartNet: A Large-scale Benchmark for Fine-grained and Hierarchical Part-level 3D Object Understandingcited by 521
  11. 2018An Algorithmic Perspective on Imitation Learningcited by 455
  12. 2019PointPainting: Sequential Fusion for 3D Object Detectioncited by 449
38 more
  1. 2016Visual Dialogcited by 377
  2. 2022Rethinking Network Design and Local Geometry in Point Cloud: A Simple Residual MLP Frameworkcited by 376
  3. 2023RealFusion: 360deg Reconstruction of Any Object from a Single Imagecited by 305
  4. 2016What do different evaluation metrics tell us about saliency models?cited by 184
  5. 2021Where2Act: From Pixels to Actions for Articulated 3D Objectscited by 179
  6. 2017PointFusion: Deep Sensor Fusion for 3D Bounding Box Estimationcited by 170
  7. 2021Multimodal Virtual Point 3D Detectioncited by 166
  8. 2021What Makes Multi-modal Learning Better than Single (Provably)cited by 151
  9. 2019Expectation-Maximization Attention Networks for Semantic Segmentationcited by 147
  10. 2022GAPartNet: Cross-Category Domain-Generalizable Object Perception and Manipulation via Generalizable and Actionable Partscited by 135
  11. 2018Grounded Human-Object Interaction Hotspots from Videocited by 128
  12. 2017AffordanceNet: An End-to-End Deep Learning Approach for Object Affordance Detectioncited by 126
  13. 2020Deep Learning for Image and Point Cloud Fusion in Autonomous Driving: A Reviewcited by 107
  14. 20213D AffordanceNet: A Benchmark for Visual Object Affordance Understandingcited by 100
  15. 2022Learning Affordance Grounding from Exocentric Imagescited by 82
  16. 2022AKB-48: A Real-World Articulated Object Knowledge Basecited by 81
  17. 2020Learning Dexterous Grasping with Object-Centric Visual Affordancescited by 80
  18. 2019Multi-view PointNet for 3D Scene Understandingcited by 76
  19. 2017Learning to Act Properly: Predicting and Explaining Affordances from Imagescited by 74
  20. 2020Learning Affordance Landscapes for Interaction Exploration in 3D Environmentscited by 65
  21. 2021EPMF: Efficient Perception-aware Multi-sensor Fusion for 3D Semantic Segmentationcited by 59
  22. 2022Factorizing Knowledge in Neural Networkscited by 59
  23. 2021AdaAfford: Learning to Adapt Manipulation Affordance for 3D Articulated Objects via Few-shot Interactionscited by 56
  24. 2021O2O-Afford: Annotation-Free Large-Scale Object-Object Affordance Learningcited by 46
  25. 2020Fusion-Aware Point Convolution for Online Semantic 3D Scene Segmentationcited by 40
  26. 2021One-Shot Object Affordance Detection in the Wildcited by 40
  27. 2019Adaptive Reconstruction Network for Weakly Supervised Referring Expression Groundingcited by 37
  28. 2022Cross-modal Learning for Image-Guided Point Cloud Shape Completioncited by 36
  29. 2022PartAfford: Part-level Affordance Discovery from 3D Objectscited by 35
  30. 2021LIF-Seg: LiDAR and Camera Image Fusion for 3D LiDAR Semantic Segmentationcited by 34
  31. 2019FuseSeg: LiDAR Point Cloud Segmentation Fusing Multi-Modal Datacited by 33
  32. 2021One-Shot Affordance Detectioncited by 32
  33. 2021Deep Learning Based 3D Segmentation: A Surveycited by 30
  34. 2022Grounded Affordance from Exocentric Viewcited by 21
  35. 2020Learning to Discretely Compose Reasoning Module Networks for Video Captioningcited by 20
  36. 2022Phrase-Based Affordance Detection via Cyclic Bilateral Interactioncited by 19
  37. 2022ImLoveNet: Misaligned Image-supported Registration Network for Low-overlap Point Cloud Pairscited by 17
  38. 2022FusionRCNN: LiDAR-Camera Fusion for Two-stage 3D Object Detectioncited by 14

50 most cited of 54 All 54 (JSON)

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2023: 6; 2024: 10; 2025: 24; 2026: 22.

62 papers.

Sort:

By area:

  1. 2023Implicit Neural Representation for Cooperative Low-light Image Enhancementcited by 92
  2. 2024AffordanceLLM: Grounding Affordance from Vision Language Modelscited by 55
  3. 2023LEMON: Learning 3D Human-Object Interaction Relation from 2D Imagescited by 49
  4. 2023One-Shot Open Affordance Learning with Foundation Modelscited by 45
  5. 2023Where2Explore: Few-shot Affordance Learning for Unseen Novel Categories of Articulated Objectscited by 33
  6. 2024Beyond the Contact: Discovering Comprehensive Affordance for 3D Objects from Pre-trained 2D Diffusion Modelscited by 32
  7. 20253D-AffordanceLLM: Harnessing Large Language Models for Open-Vocabulary Affordance Detection in 3D Worldscited by 29
  8. 2024EgoChoir: Capturing 3D Human-Object Interaction Regions from Egocentric Viewscited by 26
  9. 2024GREAT: Geometry-Intention Collaborative Inference for Open-Vocabulary 3D Object Affordance Groundingcited by 26
  10. 2023G2L: Semantically Aligned and Uniform Video Grounding via Geodesic and Game Theorycited by 21
  11. 2024Learning 2D Invariant Affordance Knowledge for 3D Affordance Groundingcited by 21
  12. 2025InteractVLM: 3D Interaction Reasoning from 2D Foundational Modelscited by 21
38 more
  1. 2025Grounding 3D Object Affordance with Language Instructions, Visual Observations and Interactionscited by 19
  2. 2023Background Activation Suppression for Weakly Supervised Object Localization and Semantic Segmentationcited by 17
  3. 2024GEAL: Generalizable 3D Affordance Learning with Cross-Modal Consistencycited by 17
  4. 2025OpenHOI: Open-World Hand-Object Interaction Synthesis with Multimodal Large Language Modelcited by 17
  5. 2025HERO: Human Reaction Generation from Videoscited by 15
  6. 2024INTRA: Interaction Relationship-aware Weakly Supervised Affordance Groundingcited by 12
  7. 2025Affordance Field Intervention: Enabling VLAs to Escape Memory Traps in Robotic Manipulationcited by 12
  8. 2024Grounding 3D Scene Affordance From Egocentric Interactionscited by 11
  9. 2025RAGNet: Large-scale Reasoning-based Affordance Segmentation Benchmark towards General Graspingcited by 11
  10. 2025How Do I Do That? Synthesizing 3D Hand Motion and Contacts for Everyday Interactionscited by 10
  11. 2025Weakly-Supervised Affordance Grounding Guided by Part-Level Semantic Priorscited by 9
  12. 2025TOUCH: Text-guided Controllable Generation of Free-Form Hand-Object Interactionscited by 8
  13. 2025Affogato: Open-Vocabulary Affordance Grounding with Automated Data Generation at Scalecited by 6
  14. 2025Task-Aware 3D Affordance Segmentation via 2D Guidance and Geometric Refinementcited by 6
  15. 2025Diffusion Models are Open-World Affordance Learners: Leveraging Generative Priors for 3D Affordance Learningcited by 5
  16. 2025DAViD: Modeling Dynamic Affordance of 3D Objects Using Pre-trained Video Diffusion Modelscited by 4
  17. 2025Multi-Keypoint Affordance Representation for Functional Dexterous Graspingcited by 4
  18. 2025AnchorDP3: 3D Affordance Guided Sparse Diffusion Policy for Robotic Manipulationcited by 4
  19. 2025GRACE: Estimating Geometry-level 3D Human-Scene Contact from 2D Imagescited by 3
  20. 2025O3Afford: One-Shot 3D Object-to-Object Affordance Grounding for Generalizable Robotic Manipulationcited by 3
  21. 2025Efficient and Scalable Monocular Human-Object Interaction Motion Reconstructioncited by 3
  22. 2024Visual-Geometric Collaborative Guidance for Affordance Learningcited by 2
  23. 2025Weakly-Supervised Learning of Dense Functional Correspondencescited by 2
  24. 2025Unlocking 3D Affordance Segmentation with 2D Semantic Knowledgecited by 2
  25. 2025Gaze Beyond the Frame: Forecasting Egocentric 3D Visual Spancited by 2
  26. 2026VideoAfford: Grounding 3D Affordance from Human-Object-Interaction Videos via Multimodal Large Language Modelcited by 2
  27. 2024Affordance Labeling and Exploration: A Manifold-Based Approachcited by 1
  28. 2025Object Affordance Recognition and Grounding via Multi-scale Cross-modal Representation Learningcited by 1
  29. 2025H2OFlow: Grounding Human-Object Affordances with 3D Generative Models and Dense Diffused Flowscited by 1
  30. 2026VAGNet: Grounding 3D Affordance from Human-Object Interactions in Videoscited by 1
  31. 2026HAMMER: Harnessing MLLM via Cross-Modal Integration for Intention-Driven 3D Affordance Groundingcited by 1
  32. 2026EXPLORE-Bench: Egocentric Scene Prediction with Long-Horizon Reasoningcited by 1
  33. 2026End-to-End Spatial-Temporal Transformer for Real-time 4D HOI Reconstructioncited by 1
  34. 2026CompassAD: Intent-Driven 3D Affordance Grounding in Functionally Competing Objectscited by 1
  35. 2026HOPE: Hand-Object Pressure Estimation from Monocular Videoscited by 1
  36. 2026LogiShot: Logically Coherent Cross-Shot Video Generationcited by 1
  37. 2026FSAG: Enhancing Human-to-Dexterous-Hand Finger-Specific Affordance Grounding via Diffusion Modelscited by 0
  38. 2026Part-Aware Open-Vocabulary 3D Affordance Grounding via Prototypical Semantic and Geometric Alignmentcited by 0

50 most cited of 62 All 62 (JSON)