mlsanityData to 7 Oct 2026

GREAT: Geometry-Intention Collaborative Inference for Open-Vocabulary 3D Object Affordance Grounding

arXiv 2411.19626 ↗First posted 29 Nov 2024cs.CV cs.AIcited by 26

Not on the map. Why?

Abstract (arXiv)

Open-Vocabulary 3D object affordance grounding aims to anticipate ``action possibilities'' regions on 3D objects with arbitrary instructions, which is crucial for robots to generically perceive real scenarios and respond to operational changes. Existing methods focus on combining images or languages that depict interactions with 3D geometries to introduce external interaction priors. However, they are still vulnerable to a limited semantic space by failing to leverage implied invariant geometries and potential interaction intentions. Normally, humans address complex tasks through multi-step reasoning and respond to diverse situations by leveraging associative and analogical thinking. In light of this, we propose GREAT (GeometRy-intEntion collAboraTive inference) for Open-Vocabulary 3D Object Affordance Grounding, a novel framework that mines the object invariant geometry attributes and performs analogically reason in potential interaction scenarios to form affordance knowledge, fully combining the knowledge with both geometries and visual contents to ground 3D object affordance. Besides, we introduce the Point Image Affordance Dataset v2 (PIADv2), the largest 3D object affordance dataset at present to support the task. Extensive experiments demonstrate the effectiveness and superiority of GREAT. The code and dataset are available at https://yawen-shao.github.io/GREAT/.

Key facts. Cited by 26 papers in the map (data through 7 Oct 2026). Builds on 49 of its 71 references that are in the map. First posted on arXiv 29 Nov 2024.

Builds on

49 of 71 references.

References · 49

  1. 2015Deep Residual Learning for Image Recognitioncited by 51,809
  2. 2021LoRA: Low-Rank Adaptation of Large Language Modelscited by 14,084
  3. 2019RoBERTa: A Robustly Optimized BERT Pretraining Approachcited by 13,266
  4. 2022Chain-of-Thought Prompting Elicits Reasoning in Large Language Modelscited by 12,302
  5. 2017PointNet++: Deep Hierarchical Feature Learning on Point Sets in a Metric Spacecited by 4,384
  6. 2022BLIP: Bootstrapping Language-Image Pre-training for Unified Vision-Language Understanding and Generationcited by 4,310
  7. 2023Tree of Thoughts: Deliberate Problem Solving with Large Language Modelscited by 3,155
  8. 2023MiniGPT-4: Enhancing Vision-Language Understanding with Advanced Large Language Modelscited by 2,689
  9. 2016V-Net: Fully Convolutional Neural Networks for Volumetric Medical Image Segmentationcited by 2,284
  10. 2022Objaverse: A Universe of Annotated 3D Objectscited by 1,522
  11. 2024How Far Are We to GPT-4V? Closing the Gap to Commercial Multimodal Models with Open-Source Suitescited by 1,033
  12. 2022Automatic Chain of Thought Prompting in Large Language Modelscited by 684
37 more
  1. 2018PartNet: A Large-scale Benchmark for Fine-grained and Hierarchical Part-level 3D Object Understandingcited by 521
  2. 2024ReKep: Spatio-Temporal Reasoning of Relational Keypoint Constraints for Robotic Manipulationcited by 305
  3. 2023Diffusion-based Generation, Optimization, and Planning in 3D Scenescited by 237
  4. 2023ManipLLM: Embodied Multimodal Large Language Model for Object-Centric Robotic Manipulationcited by 157
  5. 2024VisionLLM v2: An End-to-End Generalist Multimodal Large Language Model for Hundreds of Vision-Language Taskscited by 136
  6. 2024SceneVerse: Scaling 3D Vision-Language Learning for Grounded Scene Understandingcited by 130
  7. 20213D AffordanceNet: A Benchmark for Visual Object Affordance Understandingcited by 100
  8. 2022Learning Affordance Grounding from Exocentric Imagescited by 82
  9. 2022Benchmarking Robustness of 3D Point Cloud Recognition Against Common Corruptionscited by 78
  10. 2023LOCATE: Localize and Transfer Object Parts for Weakly Supervised Affordance Groundingcited by 64
  11. 2023Grounding 3D Object Affordance from 2D Interactions in Imagescited by 62
  12. 2024AffordanceLLM: Grounding Affordance from Vision Language Modelscited by 55
  13. 2024RAM: Retrieval-Based Affordance Transfer for Generalizable Zero-Shot Robotic Manipulationcited by 55
  14. 2023LEMON: Learning 3D Human-Object Interaction Relation from 2D Imagescited by 49
  15. 2021O2O-Afford: Annotation-Free Large-Scale Object-Object Affordance Learningcited by 46
  16. 2023One-Shot Open Affordance Learning with Foundation Modelscited by 45
  17. 2023Boosting Logical Reasoning in Large Language Models through a New Framework: The Graph of Thoughtcited by 41
  18. 2021One-Shot Object Affordance Detection in the Wildcited by 40
  19. 2022Cross-modal Learning for Image-Guided Point Cloud Shape Completioncited by 36
  20. 2023CoTDet: Affordance Knowledge Prompting for Task Driven Object Detectioncited by 36
  21. 2022PartAfford: Part-level Affordance Discovery from 3D Objectscited by 35
  22. 2021One-Shot Affordance Detectioncited by 32
  23. 2024EgoChoir: Capturing 3D Human-Object Interaction Regions from Egocentric Viewscited by 26
  24. 2021Learning Visual Affordance Grounding from Demonstration Videoscited by 25
  25. 2024Ag2Manip: Learning Novel Manipulation Skills with Agent-Agnostic Visual and Action Representationscited by 23
  26. 2018Visual Affordance and Function Understanding: A Surveycited by 20
  27. 2022Phrase-Based Affordance Detection via Cyclic Bilateral Interactioncited by 19
  28. 2023Open-Vocabulary Affordance Detection using Knowledge Distillation and Text-Point Correlationcited by 18
  29. 2023Background Activation Suppression for Weakly Supervised Object Localization and Semantic Segmentationcited by 17
  30. 2024MADiff: Motion-Aware Mamba Diffusion Models for Hand Trajectory Prediction on Egocentric Videoscited by 17
  31. 2022FusionRCNN: LiDAR-Camera Fusion for Two-stage 3D Object Detectioncited by 14
  32. 2024WorldAfford: Affordance Grounding based on Natural Language Instructionscited by 14
  33. 2024Grounding 3D Scene Affordance From Egocentric Interactionscited by 11
  34. 2024Global-Local Collaborative Inference with LLM for Lidar-Based Open-Vocabulary Detectioncited by 10
  35. 2024PEAR: Phrase-Based Hand-Object Interaction Anticipationcited by 10
  36. 2024Bidirectional Progressive Transformer for Interaction Intention Anticipationcited by 6
  37. 2024KptLLM: Unveiling the Power of Large Language Model for Keypoint Comprehensioncited by 5

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2024: 1; 2025: 9; 2026: 16.

26 papers.

Sort:

By area:

  1. 2025Affordance-R1: Reinforcement Learning for Generalizable Affordance Reasoning in Multimodal Large Language Modelcited by 21
  2. 2024SeqAfford: Sequential 3D Affordance Reasoning via Multimodal Large Language Modelcited by 18
  3. 2025HERO: Human Reaction Generation from Videoscited by 15
  4. 2025TOUCH: Text-guided Controllable Generation of Free-Form Hand-Object Interactionscited by 8
  5. 2025Digital Twin Generation from Visual Data: A Surveycited by 7
  6. 2025Task-Aware 3D Affordance Segmentation via 2D Guidance and Geometric Refinementcited by 6
  7. 2025Diffusion Models are Open-World Affordance Learners: Leveraging Generative Priors for 3D Affordance Learningcited by 5
  8. 2025A4-Agent: An Agentic Framework for Zero-Shot Affordance Reasoningcited by 4
  9. 2026Gloria: Consistent Character Video Generation via Content Anchorscited by 3
  10. 2025Unlocking 3D Affordance Segmentation with 2D Semantic Knowledgecited by 2
  11. 2026The Semantic Lifecycle in Embodied AI: Acquisition, Representation and Storage via Foundation Modelscited by 2
  12. 2026VideoAfford: Grounding 3D Affordance from Human-Object-Interaction Videos via Multimodal Large Language Modelcited by 2
14 more
  1. 2026VAGNet: Grounding 3D Affordance from Human-Object Interactions in Videoscited by 1
  2. 2026HAMMER: Harnessing MLLM via Cross-Modal Integration for Intention-Driven 3D Affordance Groundingcited by 1
  3. 2026CompassAD: Intent-Driven 3D Affordance Grounding in Functionally Competing Objectscited by 1
  4. 2026LogiShot: Logically Coherent Cross-Shot Video Generationcited by 1
  5. 2025YOLOA: Real-Time Affordance Detection via LLM Adaptercited by 0
  6. 2026FSAG: Enhancing Human-to-Dexterous-Hand Finger-Specific Affordance Grounding via Diffusion Modelscited by 0
  7. 2026Int3DNet: Scene-Motion Cross Attention Network for 3D Intention Prediction in Mixed Realitycited by 0
  8. 2026AffordMatcher: Affordance Learning in 3D Scenes from Visual Signifierscited by 0
  9. 2026VoxAfford: Multi-Scale Voxel-Token Fusion for Open-Vocabulary 3D Affordance Detectioncited by 0
  10. 2026AFUN: Towards an Affordance Foundation Model for Functionality Understandingcited by 0
  11. 2026Back on Track: Aligning Rewards and States for Reasoning in Diffusion Large Language Modelscited by 0
  12. 2026Zero-shot 2D Grounding with Novel Affordance Typescited by 0
  13. 2026UniAfford: Token-Routed Multitask Learning for Generalizable 2D-3D Affordance Perceptioncited by 0
  14. 2026Task-Adaptive Grounded 3D-Programmers Using 2D VLMscited by 0