mlsanityData to 14 Oct 2026

Capturing Fine-Grained Alignments Improves 3D Affordance Detection

arXiv 2506.19312 ↗First posted 24 Jun 2025cs.CV cs.AIcited by 0

Not on the map. Why?

Abstract (arXiv)

In this work, we address the challenge of affordance detection in 3D point clouds, a task that requires effectively capturing fine-grained alignments between point clouds and text. Existing methods often struggle to model such alignments, resulting in limited performance on standard benchmarks. A key limitation of these approaches is their reliance on simple cosine similarity between point cloud and text embeddings, which lacks the expressiveness needed for fine-grained reasoning. To address this limitation, we propose LM-AD, a novel method for affordance detection in 3D point clouds. Moreover, we introduce the Affordance Query Module (AQM), which efficiently captures fine-grained alignment between point clouds and text by leveraging a pretrained language model. We demonstrated that our method outperformed existing approaches in terms of accuracy and mean Intersection over Union on the 3D AffordanceNet dataset.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Builds on 46 of its 76 references that are in the map. First posted on arXiv 24 Jun 2025.

Builds on

46 of 76 references.

References · 46

  1. 2017Attention Is All You Needcited by 60,361
  2. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,399
  3. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,744
  4. 2023Visual Instruction Tuningcited by 7,869
  5. 2016PointNet: Deep Learning on Point Sets for 3D Classification and Segmentationcited by 5,248
  6. 2017PointNet++: Deep Hierarchical Feature Learning on Point Sets in a Metric Spacecited by 4,401
  7. 2022Flamingo: a Visual Language Model for Few-Shot Learningcited by 4,386
  8. 2024Qwen2-VL: Enhancing Vision-Language Model's Perception of the World at Any Resolutioncited by 4,234
  9. 2021Scaling Up Visual and Vision-Language Representation Learning With Noisy Text Supervisioncited by 3,864
  10. 2022Training Compute-Optimal Large Language Modelscited by 3,019
  11. 2023InstructBLIP: Towards General-purpose Vision-Language Models with Instruction Tuningcited by 2,876
  12. 2023MiniGPT-4: Enhancing Vision-Language Understanding with Advanced Large Language Modelscited by 2,711
34 more
  1. 2023Qwen-VL: A Versatile Vision-Language Model for Understanding, Localization, Text Reading, and Beyondcited by 2,224
  2. 2018Dynamic Graph CNN for Learning on Point Cloudscited by 2,223
  3. 2020DeBERTa: Decoding-enhanced BERT with Disentangled Attentioncited by 2,181
  4. 2015SegNet: A Deep Convolutional Encoder-Decoder Architecture for Image Segmentationcited by 1,487
  5. 2020Point Transformercited by 1,470
  6. 2021ViLT: Vision-and-Language Transformer Without Convolution or Region Supervisioncited by 1,230
  7. 2021SimVLM: Simple Visual Language Model Pretraining with Weak Supervisioncited by 647
  8. 2021VLMo: Unified Vision-Language Pre-Training with Mixture-of-Modality-Expertscited by 425
  9. 2018DenseFuse: A Fusion Approach to Infrared and Visible Imagescited by 180
  10. 2017AffordanceNet: An End-to-End Deep Learning Approach for Object Affordance Detectioncited by 128
  11. 2021Towards artificial general intelligence via a multimodal foundation modelcited by 112
  12. 20213D AffordanceNet: A Benchmark for Visual Object Affordance Understandingcited by 100
  13. 2022Learning Affordance Grounding from Exocentric Imagescited by 82
  14. 2024Multimodal Alignment and Fusion: A Surveycited by 74
  15. 2022BridgeTower: Building Bridges Between Encoders in Vision-Language Representation Learningcited by 56
  16. 2021Generative Zero-Shot Learning for Semantic Segmentation of 3D Point Cloudscited by 55
  17. 2024AffordanceLLM: Grounding Affordance from Vision Language Modelscited by 55
  18. 2019Predicting 3D Human Dynamics from Videocited by 51
  19. 2024Polos: Multimodal Metric Learning from Human Feedback for Image Captioningcited by 46
  20. 2023Open-Vocabulary Affordance Detection in 3D Point Cloudscited by 41
  21. 2019Transductive Zero-Shot Learning for 3D Point Cloud Classificationcited by 39
  22. 2019Zero-shot Learning of 3D Point Cloud Objectscited by 34
  23. 2023Affordance Grounding from Demonstration Video to Target Imagecited by 34
  24. 2022A4T: Hierarchical Affordance Detection for Transparent Objects Depth Reconstruction and Manipulationcited by 23
  25. 2017Predicting Human Activities Using Stochastic Grammarcited by 22
  26. 2021Cerberus Transformer: Joint Semantic, Affordance and Attribute Parsingcited by 22
  27. 2020Guided Deep Decoder: Unsupervised Image Pair Fusioncited by 20
  28. 2023Open-Vocabulary Affordance Detection using Knowledge Distillation and Text-Point Correlationcited by 18
  29. 2017Deep Affordance-grounded Sensorimotor Object Recognitioncited by 17
  30. 2023Beyond Two-Tower Matching: Learning Sparse Retrievable Cross-Interactions for Recommendationcited by 13
  31. 2020A Deep Learning Approach to Object Affordance Segmentationcited by 9
  32. 2024Multimodal Reranking for Knowledge-Intensive Visual Question Answeringcited by 6
  33. 2024I can listen but cannot read: An evaluation of two-tower multimodal systems for instrument recognitioncited by 4
  34. 2020Towards User Friendly Medication Mapping Using Entity-Boosted Two-Tower Neural Networkcited by 2

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 20253D-AffordanceLLM: Harnessing Large Language Models for Open-Vocabulary Affordance Detection in 3D Worlds
  2. 2026Part-Aware Open-Vocabulary 3D Affordance Grounding via Prototypical Semantic and Geometric Alignment
  3. 2026VoxAfford: Multi-Scale Voxel-Token Fusion for Open-Vocabulary 3D Affordance Detection
  4. 2024PAVLM: Advancing Point Cloud based Affordance Understanding Via Vision-Language Model
  5. 2023Language-Conditioned Affordance-Pose Detection in 3D Point Clouds
  6. 2026HAMMER: Harnessing MLLM via Cross-Modal Integration for Intention-Driven 3D Affordance Grounding
  7. 2025Task-Aware 3D Affordance Segmentation via 2D Guidance and Geometric Refinement
  8. 2024SeqAfford: Sequential 3D Affordance Reasoning via Multimodal Large Language Model