mlsanityData to 7 Oct 2026

Prompt-guided Scene Generation for 3D Zero-Shot Learning

arXiv 2209.14690 ↗First posted 29 Sep 2022cs.CVcited by 2

Not on the map. Why?

Abstract (arXiv)

Zero-shot learning on 3D point cloud data is a related underexplored problem compared to its 2D image counterpart. 3D data brings new challenges for ZSL due to the unavailability of robust pre-trained feature extraction models. To address this problem, we propose a prompt-guided 3D scene generation and supervision method that augments 3D data to learn the network better, exploring the complex interplay of seen and unseen objects. First, we merge point clouds of two 3D models in certain ways described by a prompt. The prompt acts like the annotation describing each 3D scene. Later, we perform contrastive learning to train our proposed architecture in an end-to-end manner. We argue that 3D scenes can relate objects more efficiently than single objects because popular language models (like BERT) can achieve high performance when objects appear in a context. Our proposed prompt-guided scene generation method encapsulates data augmentation and prompt-based annotation/captioning to improve 3D ZSL performance. We have achieved state-of-the-art ZSL and generalized ZSL performance on synthetic (ModelNet40, ModelNet10) and real-scanned (ScanOjbectNN) 3D object datasets.

Key facts. Cited by 2 papers in the map (data through 7 Oct 2026). Builds on 35 of its 42 references that are in the map. First posted on arXiv 29 Sep 2022.

Builds on

35 of 42 references.

References · 35

  1. 2015Deep Residual Learning for Image Recognitioncited by 51,809
  2. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,204
  3. 2014Adam: A Method for Stochastic Optimizationcited by 36,960
  4. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,456
  5. 2020Language Models are Few-Shot Learnerscited by 32,199
  6. 2014Very Deep Convolutional Networks for Large-Scale Image Recognitioncited by 15,267
  7. 2016PointNet: Deep Learning on Point Sets for 3D Classification and Segmentationcited by 5,223
  8. 2017PointNet++: Deep Hierarchical Feature Learning on Point Sets in a Metric Spacecited by 4,384
  9. 2013Distributed Representations of Words and Phrases and their Compositionalitycited by 3,462
  10. 2020Making Pre-trained Language Models Better Few-shot Learnerscited by 1,255
  11. 2020Exploiting Cloze Questions for Few Shot Text Classification and Natural Language Inferencecited by 934
  12. 2021Multimodal Few-Shot Learning with Frozen Language Modelscited by 657
23 more
  1. 2017Zero-Shot Learning -- A Comprehensive Evaluation of the Good, the Bad and the Uglycited by 536
  2. 2019Revisiting Point Cloud Classification: A New Benchmark Dataset and Classification Model on Real-World Datacited by 532
  3. 2017Feature Generating Networks for Zero-Shot Learningcited by 257
  4. 2013Zero-Shot Learning Through Cross-Modal Transfercited by 237
  5. 2018Generalized Zero- and Few-Shot Learning via Aligned Variational Autoencoderscited by 206
  6. 2014Evaluation of Output Embeddings for Fine-Grained Image Classificationcited by 166
  7. 2015Label-Embedding for Image Classificationcited by 164
  8. 2016Learning a Deep Embedding Model for Zero-Shot Learningcited by 153
  9. 2016Latent Embeddings for Zero-shot Classificationcited by 129
  10. 2016Synthesized Classifiers for Zero-Shot Learningcited by 113
  11. 2017A Generative Model For Zero Shot Learning Using Conditional Variational Autoencoderscited by 64
  12. 2020Leveraging Seen and Unseen Semantic Relationships for Generative Zero-Shot Learningcited by 64
  13. 2018Generative Dual Adversarial Network for Generalized Zero-shot Learningcited by 61
  14. 2021Generative Zero-Shot Learning for Semantic Segmentation of 3D Point Cloudscited by 55
  15. 2015Ridge Regression, Hubness, and Zero-Shot Learningcited by 44
  16. 2019Rethinking Zero-Shot Learning: A Conditional Visual Classification Perspectivecited by 42
  17. 2019Transductive Zero-Shot Learning for 3D Point Cloud Classificationcited by 39
  18. 2017A Simple Exponential Family Framework for Zero-Shot Learningcited by 38
  19. 2021Zero-Shot Learning on 3D Point Cloud Objects and Beyondcited by 38
  20. 2019Zero-shot Learning of 3D Point Cloud Objectscited by 34
  21. 2019Mitigating the Hubness Problem for Zero-Shot Learning of 3D Objectscited by 24
  22. 2017Zero-Shot Recognition using Dual Visual-Semantic Mapping Pathscited by 20
  23. 2019A Generative Framework for Zero-Shot Learning with Adversarial Domain Adaptationcited by 8

Built on by

Cited by year

Citing papers per year: 2023: 1; 2024: 1.

2 papers.

  1. 2024Automatic Scene Generation: State-of-the-Art Techniques, Models, Datasets, Challenges, and Future Prospectscited by 2
  2. 2023ChatGPT-guided Semantics for Zero-shot Learningcited by 1