Part-Aware Open-Vocabulary 3D Affordance Grounding via Prototypical Semantic and Geometric Alignment
Not on the map. Why?
Abstract (arXiv)
Grounding natural language questions to functionally relevant regions in 3D objects -- termed language-driven 3D affordance grounding -- is essential for embodied intelligence and human-AI interaction. Existing methods, while progressing from label-based to language-driven approaches, still face challenges in open-vocabulary generalization, fine-grained geometric alignment, and part-level semantic consistency. To address these issues, we propose a novel two-stage cross-modal framework that enhances both semantic and geometric representations for open-vocabulary 3D affordance grounding. In the first stage, large language models generate part-aware instructions to recover missing semantics, enabling the model to link semantically similar affordances. In the second stage, we introduce two key components: Affordance Prototype Aggregation (APA), which captures cross-object geometric consistency for each affordance, and Intra-Object Relational Modeling (IORM), which refines geometric differentiation within objects to support precise semantic alignment. We validate the effectiveness of our method through extensive experiments on a newly introduced benchmark, as well as two existing benchmarks, demonstrating superior performance in comparison with existing methods.
Key facts. No citing papers in the map yet (data through 14 Oct 2026). Builds on 40 of its 48 references that are in the map. First posted on arXiv 18 Mar 2026.
Builds on
40 of 48 references.
References · 40
- 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,399
- 2019RoBERTa: A Robustly Optimized BERT Pretraining Approachcited by 13,372
- 2017PointNet++: Deep Hierarchical Feature Learning on Point Sets in a Metric Spacecited by 4,401
- 2015ShapeNet: An Information-Rich 3D Model Repositorycited by 2,897
- 2016V-Net: Fully Convolutional Neural Networks for Volumetric Medical Image Segmentationcited by 2,292
- 2023PaLM-E: An Embodied Multimodal Language Modelcited by 2,071
- 2021Ego4D: Around the World in 3,000 Hours of Egocentric Videocited by 1,556
- 2022OpenScene: 3D Scene Understanding with Open Vocabulariescited by 559
- 2023LERF: Language Embedded Radiance Fieldscited by 543
- 2017Structure-measure: A New Way to Evaluate Foreground Mapscited by 480
- 2019ScanRefer: 3D Object Localization in RGB-D Scans using Natural Languagecited by 452
- 2023LangSplat: 3D Language Gaussian Splattingcited by 415
28 more
- 2022Visual Language Maps for Robot Navigationcited by 365
- 2021ScanQA: 3D Question Answering for Spatial Scene Understandingcited by 346
- 2022SQA3D: Situated Question Answering in 3D Scenescited by 314
- 2023OpenMask3D: Open-Vocabulary 3D Instance Segmentationcited by 289
- 2022ULIP: Learning a Unified Representation of Language, Images, and Point Clouds for 3D Understandingcited by 267
- 2023GRES: Generalized Referring Expression Segmentationcited by 254
- 2023OpenShape: Scaling Up 3D Shape Representation Towards Open-World Understandingcited by 184
- 2022PLA: Language-Driven Open-Vocabulary 3D Scene Understandingcited by 177
- 2021Referring Transformer: A One-step Approach to Multi-task Visual Groundingcited by 143
- 2024ShapeLLM: Universal 3D Object Understanding for Embodied Interactioncited by 137
- 2021Bottom Up Top Down Detection Transformers for Language Grounding in Images and Point Cloudscited by 128
- 2022Language Conditioned Spatial Relation Reasoning for 3D Object Groundingcited by 122
- 20223D-SPS: Single-Stage 3D Visual Grounding via Referred Point Progressive Selectioncited by 110
- 20213D AffordanceNet: A Benchmark for Visual Object Affordance Understandingcited by 100
- 2023OpenIns3D: Snap and Lookup for 3D Open-vocabulary Instance Segmentationcited by 82
- 2023Open-Vocabulary Point-Cloud Object Detection without 3D Annotationcited by 81
- 2023Grounding 3D Object Affordance from 2D Interactions in Imagescited by 62
- 2023Open-Fusion: Real-time Open-Vocabulary 3D Mapping and Queryable Scene Representationcited by 54
- 2024POP-3D: Open-Vocabulary 3D Occupancy Prediction from Imagescited by 52
- 2023Open-Vocabulary Affordance Detection in 3D Point Cloudscited by 41
- 2022Cross-modal Learning for Image-Guided Point Cloud Shape Completioncited by 36
- 2021Trans4Trans: Efficient Transformer for Transparent Object Segmentation to Help Visually Impaired People Navigate in the Real Worldcited by 30
- 20253D-AffordanceLLM: Harnessing Large Language Models for Open-Vocabulary Affordance Detection in 3D Worldscited by 30
- 2024Learning 2D Invariant Affordance Knowledge for 3D Affordance Groundingcited by 22
- 2025Grounding 3D Object Affordance with Language Instructions, Visual Observations and Interactionscited by 19
- 2024SeqAfford: Sequential 3D Affordance Reasoning via Multimodal Large Language Modelcited by 18
- 2024ImOV3D: Learning Open-Vocabulary Point Clouds 3D Object Detection from Only 2D Imagescited by 17
- 2025Object Affordance Recognition and Grounding via Multi-scale Cross-modal Representation Learningcited by 1
Built on by
Not cited yet.
Similar, not cited
Close in wording, no citation either way
- 2026AffordAny: Open-World 3D Affordance Grounding from Monocular RGB Images via Vision-Language-Guided Geometric Reasoning
- 2026HAMMER: Harnessing MLLM via Cross-Modal Integration for Intention-Driven 3D Affordance Grounding
- 2026Grounding by Remembering: Cross-Scene and In-Scene Memory for 3D Functional Affordances
- 2026ThinkAfford: Affordance-Centric Reasoning for Fine-Grained 3D Grounding in Cluttered Scenes
- 2025Task-Aware 3D Affordance Segmentation via 2D Guidance and Geometric Refinement
- 2026UniAfford: Token-Routed Multitask Learning for Generalizable 2D-3D Affordance Perception
- 2024WorldAfford: Affordance Grounding based on Natural Language Instructions
- 2025AffordBot: 3D Fine-grained Embodied Reasoning via Multimodal Large Language Models