mlsanityData to 7 Oct 2026

Zero-Shot Learning on 3D Point Cloud Objects and Beyond

arXiv 2104.04980 ↗First posted 11 Apr 2021cs.CVcited by 38

Not on the map. Why?

Abstract (arXiv)

Zero-shot learning, the task of learning to recognize new classes not seen during training, has received considerable attention in the case of 2D image classification. However, despite the increasing ubiquity of 3D sensors, the corresponding 3D point cloud classification problem has not been meaningfully explored and introduces new challenges. In this paper, we identify some of the challenges and apply 2D Zero-Shot Learning (ZSL) methods in the 3D domain to analyze the performance of existing models. Then, we propose a novel approach to address the issues specific to 3D ZSL. We first present an inductive ZSL process and then extend it to the transductive ZSL and Generalized ZSL (GZSL) settings for 3D point cloud classification. To this end, a novel loss function is developed that simultaneously aligns seen semantics with point cloud features and takes advantage of unlabeled test data to address some known issues (e.g., the problems of domain adaptation, hubness, and data bias). While designed for the particularities of 3D point cloud classification, the method is shown to also be applicable to the more common use-case of 2D image classification. An extensive set of experiments is carried out, establishing state-of-the-art for ZSL and GZSL on synthetic (ModelNet40, ModelNet10, McGill) and real (ScanObjectNN) 3D point cloud datasets.

Key facts. Cited by 38 papers in the map (data through 7 Oct 2026). Builds on 55 of its 81 references that are in the map. First posted on arXiv 11 Apr 2021.

Builds on

55 of 81 references.

References · 55

  1. 2015Deep Residual Learning for Image Recognitioncited by 51,809
  2. 2014Adam: A Method for Stochastic Optimizationcited by 36,960
  3. 2014ImageNet Large Scale Visual Recognition Challengecited by 8,441
  4. 2016PointNet: Deep Learning on Point Sets for 3D Classification and Segmentationcited by 5,223
  5. 2017mixup: Beyond Empirical Risk Minimizationcited by 5,155
  6. 2017PointNet++: Deep Hierarchical Feature Learning on Point Sets in a Metric Spacecited by 4,384
  7. 2013Efficient Estimation of Word Representations in Vector Spacecited by 4,346
  8. 2013Distributed Representations of Words and Phrases and their Compositionalitycited by 3,462
  9. 2015FaceNet: A Unified Embedding for Face Recognition and Clusteringcited by 2,833
  10. 2018Dynamic Graph CNN for Learning on Point Cloudscited by 2,220
  11. 2018PointConv: Deep Convolutional Networks on 3D Point Cloudscited by 656
  12. 2015Multi-view Convolutional Neural Networks for 3D Shape Recognitioncited by 564
38 more
  1. 2017Zero-Shot Learning -- A Comprehensive Evaluation of the Good, the Bad and the Uglycited by 536
  2. 2019Revisiting Point Cloud Classification: A New Benchmark Dataset and Classification Model on Real-World Datacited by 532
  3. 2018SpiderCNN: Deep Learning on Point Sets with Parameterized Convolutional Filterscited by 261
  4. 2017Feature Generating Networks for Zero-Shot Learningcited by 257
  5. 2018SO-Net: Self-Organizing Network for Point Cloud Analysiscited by 210
  6. 2018Generalized Zero- and Few-Shot Learning via Aligned Variational Autoencoderscited by 206
  7. 2014Evaluation of Output Embeddings for Fine-Grained Image Classificationcited by 166
  8. 2015Label-Embedding for Image Classificationcited by 164
  9. 2019f-VAEGAN-D2: A Feature Generating Framework for Any-Shot Learningcited by 164
  10. 2016Learning a Deep Embedding Model for Zero-Shot Learningcited by 153
  11. 2016An Empirical Study and Analysis of Generalized Zero-Shot Learning for Object Recognition in the Wildcited by 146
  12. 2016Latent Embeddings for Zero-shot Classificationcited by 129
  13. 2016Synthesized Classifiers for Zero-Shot Learningcited by 113
  14. 2020PointMixup: Augmentation for Point Cloudscited by 104
  15. 2018Local Spectral Graph Convolution for Point Set Feature Learningcited by 84
  16. 2019Leveraging the Invariant Side of Generative Zero-Shot Learningcited by 83
  17. 2017Multi-Label Zero-Shot Learning with Structured Knowledge Graphscited by 72
  18. 2018Triplet-Center Loss for Multi-View 3D Object Retrievalcited by 69
  19. 2020PointAugment: an Auto-Augmentation Framework for Point Cloud Classificationcited by 65
  20. 2019Transferable Contrastive Network for Generalized Zero-Shot Learningcited by 64
  21. 2020Leveraging Seen and Unseen Semantic Relationships for Generative Zero-Shot Learningcited by 64
  22. 2018Generative Dual Adversarial Network for Generalized Zero-shot Learningcited by 61
  23. 2014Improving zero-shot learning by mitigating the hubness problemcited by 53
  24. 2015Transductive Multi-view Zero-Shot Learningcited by 53
  25. 2015Ridge Regression, Hubness, and Zero-Shot Learningcited by 44
  26. 2018Transductive Unbiased Embedding for Zero-Shot Learningcited by 44
  27. 2019Rethinking Zero-Shot Learning: A Conditional Visual Classification Perspectivecited by 42
  28. 2019Transductive Zero-Shot Learning for 3D Point Cloud Classificationcited by 39
  29. 2017A Simple Exponential Family Framework for Zero-Shot Learningcited by 38
  30. 2019Zero-shot Learning of 3D Point Cloud Objectscited by 34
  31. 2017A Unified approach for Conventional Zero-shot, Generalized Zero-shot and Few-shot Learningcited by 28
  32. 2019A Theoretically Sound Upper Bound on the Triplet Loss for Improving the Efficiency of Deep Distance Metric Learningcited by 25
  33. 2019Mitigating the Hubness Problem for Zero-Shot Learning of 3D Objectscited by 24
  34. 2018Progressive Ensemble Networks for Zero-Shot Recognitioncited by 23
  35. 2018Towards Effective Deep Embedding for Zero-Shot Learningcited by 19
  36. 20193D Object Instance Recognition and Pose Estimation Using Triplet Loss with Dynamic Margincited by 14
  37. 2018Angular Triplet-Center Loss for Multi-view 3D Shape Retrievalcited by 11
  38. 2018Deep Multiple Instance Learning for Zero-shot Image Taggingcited by 8

50 most cited of 55 All 55 (JSON)

Built on by

Cited by year

Citing papers per year: 2021: 3; 2022: 10; 2023: 12; 2024: 9; 2025: 4.

38 papers.

Sort:

By area:

  1. 2022OpenScene: 3D Scene Understanding with Open Vocabulariescited by 556
  2. 2021PointCLIP: Point Cloud Understanding by CLIPcited by 461
  3. 2022Language-Grounded Indoor 3D Semantic Segmentation in the Wildcited by 243
  4. 2022PointCLIP V2: Prompting CLIP and GPT for Powerful 3D Open-world Learningcited by 186
  5. 2023CLIP2Scene: Towards Label-efficient 3D Scene Understanding by CLIPcited by 178
  6. 2022CLIP2Point: Transfer CLIP to Point Cloud Classification with Image-Depth Pre-trainingcited by 159
  7. 2023CLIP-FO3D: Learning Free Open-world 3D Scene Representations from 2D Dense CLIPcited by 89
  8. 2023CLIP2: Contrastive Language-Image-Point Pretraining from Real-World Point Cloud Datacited by 69
  9. 2021Generative Zero-Shot Learning for Semantic Segmentation of 3D Point Cloudscited by 55
  10. 2022LidarCLIP or: How I Learned to Talk to Point Cloudscited by 33
  11. 2024PointAD: Comprehending 3D Anomalies from Points and Pixels for Zero-shot 3D Anomaly Detectioncited by 29
  12. 2023PartDistill: 3D Shape Part Segmentation by Vision-Language Model Distillationcited by 27
26 more
  1. 2022Open-Vocabulary 3D Detection via Image-level Class and Debiased Cross-modal Contrastive Learningcited by 23
  2. 2023OpenSight: A Simple Open-Vocabulary Framework for LiDAR-Based Object Detectioncited by 22
  3. 2023See More and Know More: Zero-shot Point Cloud Segmentation via Multi-modal Visual Datacited by 21
  4. 2022Few-shot Class-incremental Learning for 3D Point Cloud Objectscited by 13
  5. 2023Geometrically-driven Aggregation for Zero-shot 3D Point Cloud Understandingcited by 13
  6. 2024Open-Vocabulary 3D Semantic Segmentation with Text-to-Image Diffusion Modelscited by 12
  7. 2023MV-CLIP: Multi-View CLIP for Zero-shot 3D Shape Recognitioncited by 11
  8. 2022Zero-shot point cloud segmentation by transferring geometric primitivescited by 9
  9. 2024Towards Zero-shot 3D Anomaly Localizationcited by 8
  10. 2021Learning without Forgetting for 3D Point Cloud Objectscited by 7
  11. 2023Reliability-Adaptive Consistency Regularization for Weakly-Supervised Point Cloud Segmentationcited by 5
  12. 2024Exploiting GPT-4 Vision for Zero-shot Point Cloud Understandingcited by 5
  13. 2024On the Element-Wise Representation and Reasoning in Zero-Shot Image Recognition: A Systematic Surveycited by 5
  14. 2023Open-Pose 3D Zero-Shot Learning: Benchmark and Challengescited by 4
  15. 2023ExpPoint-MAE: Better interpretability and performance for self-supervised point cloud transformerscited by 3
  16. 2022Prompt-guided Scene Generation for 3D Zero-Shot Learningcited by 2
  17. 2025Generalized Zero-Shot Learning for Point Cloud Segmentation with Evidence-Based Dynamic Calibrationcited by 2
  18. 2025MCL-AD: Multimodal Collaboration Learning for Zero-Shot 3D Anomaly Detectioncited by 2
  19. 2022Text2Model: Text-based Model Induction for Zero-shot Image Classificationcited by 1
  20. 2023ChatGPT-guided Semantics for Zero-shot Learningcited by 1
  21. 2024Triple Point Maskingcited by 1
  22. 2025PointAD+: Learning Hierarchical Representations for Zero-shot 3D Anomaly Detectioncited by 1
  23. 2024ESP-Zero: Unsupervised enhancement of zero-shot classification for Extremely Sparse Point cloudcited by 0
  24. 2024Curvature Diversity-Driven Deformation and Domain Alignment for Point Cloudcited by 0
  25. 2024UniRiT: Towards Few-Shot Non-Rigid Point Cloud Registrationcited by 0
  26. 2025Towards Training-Free Open-World Classification with 3D Generative Modelscited by 0