mlsanityData to 7 Oct 2026

Open-Vocabulary Affordance Detection in 3D Point Clouds

arXiv 2303.02401 ↗First posted 4 Mar 2023cs.RO cs.AI cs.CVcited by 41

Not on the map. Why?

Abstract (arXiv)

Affordance detection is a challenging problem with a wide variety of robotic applications. Traditional affordance detection methods are limited to a predefined set of affordance labels, hence potentially restricting the adaptability of intelligent robots in complex and dynamic environments. In this paper, we present the Open-Vocabulary Affordance Detection (OpenAD) method, which is capable of detecting an unbounded number of affordances in 3D point clouds. By simultaneously learning the affordance text and the point feature, OpenAD successfully exploits the semantic relationships between affordances. Therefore, our proposed method enables zero-shot detection and can be able to detect previously unseen affordances without a single annotation example. Intensive experimental results show that OpenAD works effectively on a wide range of affordance detection setups and outperforms other baselines by a large margin. Additionally, we demonstrate the practicality of the proposed OpenAD in real-world robotic applications with a fast inference speed (~100ms). Our project is available at https://openad2023.github.io.

Key facts. Cited by 41 papers in the map (data through 7 Oct 2026). Builds on 49 of its 88 references that are in the map. First posted on arXiv 4 Mar 2023.

Builds on

49 of 88 references.

References · 49

  1. 2017Attention Is All You Needcited by 59,932
  2. 2015Deep Residual Learning for Image Recognitioncited by 51,809
  3. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,204
  4. 2014Adam: A Method for Stochastic Optimizationcited by 36,960
  5. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,456
  6. 2014Very Deep Convolutional Networks for Large-Scale Image Recognitioncited by 15,267
  7. 2017PointNet++: Deep Hierarchical Feature Learning on Point Sets in a Metric Spacecited by 4,384
  8. 2013Distributed Representations of Words and Phrases and their Compositionalitycited by 3,462
  9. 2015ShapeNet: An Information-Rich 3D Model Repositorycited by 2,880
  10. 2018Dynamic Graph CNN for Learning on Point Cloudscited by 2,220
  11. 2018Unsupervised Feature Learning via Non-Parametric Instance-level Discriminationcited by 1,838
  12. 2020Point Transformercited by 1,462
37 more
  1. 2017VoxelNet: End-to-End Learning for Point Cloud Based 3D Object Detectioncited by 1,323
  2. 2022Language-driven Semantic Segmentationcited by 711
  3. 2019Deep Learning for 3D Point Clouds: A Surveycited by 557
  4. 2022OpenScene: 3D Scene Understanding with Open Vocabulariescited by 556
  5. 2021Image Segmentation Using Text and Image Promptscited by 509
  6. 2021Scaling Open-Vocabulary Image Segmentation with Image-Level Labelscited by 471
  7. 2022Language-Grounded Indoor 3D Semantic Segmentation in the Wildcited by 243
  8. 2015Structural-RNN: Deep Learning on Spatio-Temporal Graphscited by 234
  9. 2015DeepDriving: Learning Affordance for Direct Perception in Autonomous Drivingcited by 229
  10. 2021A Simple Baseline for Open-Vocabulary Semantic Segmentation with Pre-trained Vision-language Modelcited by 223
  11. 2017AffordanceNet: An End-to-End Deep Learning Approach for Object Affordance Detectioncited by 126
  12. 2017Grasp Pose Detection in Point Cloudscited by 107
  13. 20213D AffordanceNet: A Benchmark for Visual Object Affordance Understandingcited by 100
  14. 2022Learning Affordance Grounding from Exocentric Imagescited by 82
  15. 2020Learning Dexterous Grasping with Object-Centric Visual Affordancescited by 80
  16. 2021Affordance Transfer Learning for Human-Object Interaction Detectioncited by 79
  17. 2017Learning to Act Properly: Predicting and Explaining Affordances from Imagescited by 74
  18. 2019Putting Humans in a Scene: Learning Affordance in 3D Indoor Environmentscited by 72
  19. 2022Open-world Semantic Segmentation via Contrasting and Clustering Vision-Language Embeddingcited by 67
  20. 2020Learning Affordance Landscapes for Interaction Exploration in 3D Environmentscited by 65
  21. 2021Generative Zero-Shot Learning for Semantic Segmentation of 3D Point Cloudscited by 55
  22. 2021O2O-Afford: Annotation-Free Large-Scale Object-Object Affordance Learningcited by 46
  23. 2021One-Shot Object Affordance Detection in the Wildcited by 40
  24. 2019Transductive Zero-Shot Learning for 3D Point Cloud Classificationcited by 39
  25. 2019Zero-shot Learning of 3D Point Cloud Objectscited by 34
  26. 2021One-Shot Affordance Detectioncited by 32
  27. 2022One-Shot Transfer of Affordance Regions? AffCorrs!cited by 31
  28. 2022A4T: Hierarchical Affordance Detection for Transparent Objects Depth Reconstruction and Manipulationcited by 23
  29. 2017Predicting Human Activities Using Stochastic Grammarcited by 22
  30. 2021Cerberus Transformer: Joint Semantic, Affordance and Attribute Parsingcited by 22
  31. 2019Affordance Learning for End-to-End Visuomotor Robot Controlcited by 18
  32. 2017Deep Affordance-grounded Sensorimotor Object Recognitioncited by 17
  33. 2017See the Glass Half Full: Reasoning about Liquid Containers, their Volume and Contentcited by 11
  34. 2016What Can I Do Around Here? Deep Functional Scene Understanding for Cognitive Robotscited by 9
  35. 2020A Deep Learning Approach to Object Affordance Segmentationcited by 9
  36. 2022Machine Learning-based Framework for Optimally Solving the Analytical Inverse Kinematics for Redundant Manipulatorscited by 9
  37. 2022Singularity Avoidance with Application to Online Trajectory Optimization for Serial Manipulatorscited by 7

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2023: 4; 2024: 14; 2025: 12; 2026: 11.

41 papers.

Sort:

By area:

  1. 2023HOI-Diff: Text-Driven Synthesis of 3D Human-Object Interactions using Diffusion Modelscited by 89
  2. 2023One-Shot Open Affordance Learning with Foundation Modelscited by 45
  3. 2024Learning Precise Affordances from Egocentric Videos for Robotic Manipulationcited by 34
  4. 2024A Survey of Embodied Learning for Object-Centric Robotic Manipulationcited by 34
  5. 2023Grasp-Anything: Large-scale Grasp Dataset from Foundation Modelscited by 33
  6. 20253D-AffordanceLLM: Harnessing Large Language Models for Open-Vocabulary Affordance Detection in 3D Worldscited by 29
  7. 2024Language-driven Grasp Detectioncited by 26
  8. 2025Grounding 3D Object Affordance with Language Instructions, Visual Observations and Interactionscited by 19
  9. 2024Language-Driven 6-DoF Grasp Detection Using Negative Prompt Guidancecited by 16
  10. 2024WorldAfford: Affordance Grounding based on Natural Language Instructionscited by 14
  11. 2023ZEETAD: Adapting Pretrained Vision-Language Model for Zero-Shot End-to-End Temporal Action Detectioncited by 12
  12. 2024INTRA: Interaction Relationship-aware Weakly Supervised Affordance Groundingcited by 12
29 more
  1. 2024PreAfford: Universal Affordance-Based Pre-Grasping for Diverse Objects and Environmentscited by 11
  2. 2025Generating 6DoF Object Manipulation Trajectories from Action Description in Egocentric Visioncited by 11
  3. 2024Text-driven Affordance Learning from Egocentric Visioncited by 9
  4. 2024Lightweight Language-driven Grasp Detection using Conditional Consistency Modelcited by 9
  5. 2025Weakly-Supervised Affordance Grounding Guided by Part-Level Semantic Priorscited by 9
  6. 2025Digital Twin Generation from Visual Data: A Surveycited by 7
  7. 2025Affogato: Open-Vocabulary Affordance Grounding with Automated Data Generation at Scalecited by 6
  8. 2024Robotic-CLIP: Fine-tuning CLIP on Action Data for Robotic Applicationscited by 5
  9. 2025Diffusion Models are Open-World Affordance Learners: Leveraging Generative Priors for 3D Affordance Learningcited by 5
  10. 2025AnchorDP3: 3D Affordance Guided Sparse Diffusion Policy for Robotic Manipulationcited by 4
  11. 2024GraspMamba: A Mamba-based Language-driven Grasp Detection Framework with Hierarchical Feature Learningcited by 3
  12. 2025Afford-X: Generalizable and Slim Affordance Reasoning for Task-oriented Manipulationcited by 3
  13. 2025Interpretable Affordance Detection on 3D Point Clouds with Probabilistic Prototypescited by 3
  14. 2024Language-driven Grasp Detection with Mask-guided Attentioncited by 2
  15. 2026VideoAfford: Grounding 3D Affordance from Human-Object-Interaction Videos via Multimodal Large Language Modelcited by 2
  16. 2024Deep-Wide Learning Assistance for Insect Pest Classificationcited by 1
  17. 2026CompassAD: Intent-Driven 3D Affordance Grounding in Functionally Competing Objectscited by 1
  18. 2024DAP: Diffusion-based Affordance Prediction for Multi-modality Storagecited by 0
  19. 2025Capturing Fine-Grained Alignments Improves 3D Affordance Detectioncited by 0
  20. 2025OpenTie: Open-vocabulary Sequential Rebar Tying Systemcited by 0
  21. 2026Int3DNet: Scene-Motion Cross Attention Network for 3D Intention Prediction in Mixed Realitycited by 0
  22. 2026Part-Aware Open-Vocabulary 3D Affordance Grounding via Prototypical Semantic and Geometric Alignmentcited by 0
  23. 2026AffordMatcher: Affordance Learning in 3D Scenes from Visual Signifierscited by 0
  24. 2026AffordSim: A Scalable Data Generator and Benchmark for Affordance-Aware Robotic Manipulationcited by 0
  25. 2026VoxAfford: Multi-Scale Voxel-Token Fusion for Open-Vocabulary 3D Affordance Detectioncited by 0
  26. 2026AFUN: Towards an Affordance Foundation Model for Functionality Understandingcited by 0
  27. 2026Beyond Visual Grasping: Benchmarking Complex Grasping from Detection to Executioncited by 0
  28. 2026Zero-shot 2D Grounding with Novel Affordance Typescited by 0
  29. 2026Gripper-aware Vision Language Action Modelscited by 0