mlsanityData to 7 Oct 2026

MV-CLIP: Multi-View CLIP for Zero-shot 3D Shape Recognition

arXiv 2311.18402 ↗First posted 30 Nov 2023cs.CVcited by 11

Not on the map. Why?

Abstract (arXiv)

Large-scale pre-trained models have demonstrated impressive performance in vision and language tasks within open-world scenarios. Due to the lack of comparable pre-trained models for 3D shapes, recent methods utilize language-image pre-training to realize zero-shot 3D shape recognition. However, due to the modality gap, pretrained language-image models are not confident enough in the generalization to 3D shape recognition. Consequently, this paper aims to improve the confidence with view selection and hierarchical prompts. Leveraging the CLIP model as an example, we employ view selection on the vision side by identifying views with high prediction confidence from multiple rendered views of a 3D shape. On the textual side, the strategy of hierarchical prompts is proposed for the first time. The first layer prompts several classification candidates with traditional class-level descriptions, while the second layer refines the prediction based on function-level descriptions or further distinctions between the candidates. Remarkably, without the need for additional training, our proposed method achieves impressive zero-shot 3D classification accuracies of 84.44%, 91.51%, and 66.17% on ModelNet40, ModelNet10, and ShapeNet Core55, respectively. Furthermore, we will make the code publicly available to facilitate reproducibility and further research in this area.

Key facts. Cited by 11 papers in the map (data through 7 Oct 2026). Builds on 48 of its 69 references that are in the map. First posted on arXiv 30 Nov 2023.

Builds on

48 of 69 references.

References · 48

  1. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,204
  2. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,456
  3. 2020Language Models are Few-Shot Learnerscited by 32,199
  4. 2015ShapeNet: An Information-Rich 3D Model Repositorycited by 2,880
  5. 2021Learning to Prompt for Vision-Language Modelscited by 2,269
  6. 2021Pre-train, Prompt, and Predict: A Systematic Survey of Prompting Methods in Natural Language Processingcited by 2,222
  7. 2022Visual Prompt Tuningcited by 1,674
  8. 2022Objaverse: A Universe of Annotated 3D Objectscited by 1,522
  9. 2020Point Transformercited by 1,462
  10. 2022Conditional Prompt Learning for Vision-Language Modelscited by 1,427
  11. 2019How Can We Know What Language Models Know?cited by 953
  12. 2019Universal Adversarial Triggers for Attacking and Analyzing NLPcited by 709
36 more
  1. 2022MaPLe: Multi-modal Prompt Learningcited by 685
  2. 2021Point-BERT: Pre-training 3D Point Cloud Transformers with Masked Point Modelingcited by 647
  3. 2015Multi-view Convolutional Neural Networks for 3D Shape Recognitioncited by 564
  4. 2019Revisiting Point Cloud Classification: A New Benchmark Dataset and Classification Model on Real-World Datacited by 532
  5. 2021PointCLIP: Point Cloud Understanding by CLIPcited by 461
  6. 2021SLIP: Self-supervision meets Language-Image Pre-trainingcited by 434
  7. 2022Exploring Visual Prompts for Adapting Large-Scale Modelscited by 271
  8. 2022Point-M2AE: Multi-scale Masked Autoencoders for Hierarchical Point Cloud Pre-trainingcited by 263
  9. 2022What does a platypus look like? Generating customized prompts for zero-shot image classificationcited by 263
  10. 2022ULIP: Learning a Unified Representation of Language, Images, and Point Clouds for 3D Understandingcited by 262
  11. 2023ULIP-2: Towards Scalable Multimodal Pre-training for 3D Understandingcited by 199
  12. 2022PointCLIP V2: Prompting CLIP and GPT for Powerful 3D Open-world Learningcited by 186
  13. 2023OpenShape: Scaling Up 3D Shape Representation Towards Open-World Understandingcited by 183
  14. 2023Contrast with Reconstruct: Contrastive 3D Representation Learning Guided by Generative Pretrainingcited by 162
  15. 2022CLIP2Point: Transfer CLIP to Point Cloud Classification with Image-Depth Pre-trainingcited by 159
  16. 2023Prompt, Generate, then Cache: Cascade of Foundation Models makes Strong Few-shot Learnerscited by 152
  17. 2021Revisiting Point Cloud Shape Classification with a Simple and Effective Baselinecited by 145
  18. 2022Learning 3D Representations from 2D Pre-trained Models via Image-to-Point Masked Autoencoderscited by 143
  19. 2020MVTN: Multi-View Transformation Network for 3D Shape Recognitioncited by 140
  20. 2023CLIP-FO3D: Learning Free Open-world 3D Scene Representations from 2D Dense CLIPcited by 89
  21. 2023CHiLS: Zero-Shot Image Classification with Hierarchical Label Setscited by 79
  22. 2023CLIP goes 3D: Leveraging Prompt Tuning for Language Grounded 3D Recognitioncited by 77
  23. 2023ViewRefer: Grasp the Multi-view Knowledge for 3D Visual Grounding with GPT and Prototype Guidancecited by 70
  24. 2023CLIP2: Contrastive Language-Image-Point Pretraining from Real-World Point Cloud Datacited by 69
  25. 2023Joint-MAE: 2D-3D Joint Masked Autoencoders for 3D Point Cloud Pre-trainingcited by 52
  26. 2021Zero-Shot Learning on 3D Point Cloud Objects and Beyondcited by 38
  27. 2019Zero-shot Learning of 3D Point Cloud Objectscited by 34
  28. 2022Point-to-Box Network for Accurate Object Detection via Single Point Supervisioncited by 31
  29. 2018A Deeper Look at 3D Shape Classifierscited by 25
  30. 2023DiffCLIP: Leveraging Stable Diffusion for Language Grounded 3D Classificationcited by 23
  31. 2022Hybrid Routing Transformer for Zero-Shot Learningcited by 20
  32. 2023MSINet: Twins Contrastive Search of Multi-Scale Interaction for Object ReIDcited by 16
  33. 2023Sketch-A-Shape: Zero-Shot Sketch-to-3D Shape Generationcited by 14
  34. 2023Beyond First Impressions: Integrating Joint Multi-modal Cues for Comprehensive 3D Representationcited by 12
  35. 2022MVTN: Learning Multi-View Transformations for 3D Understandingcited by 7
  36. 2020Learning to Detect Head Movement in Unconstrained Remote Gaze Estimation in the Wildcited by 4

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2024: 5; 2025: 3; 2026: 3.

11 papers.

  1. 2024Duoduo CLIP: Efficient 3D Understanding with Multi-View Imagescited by 12
  2. 2024The Bare Necessities: Designing Simple, Effective Open-Vocabulary Scene Graphscited by 11
  3. 2024Multi-View and Multi-Scale Alignment for Contrastive Language-Image Pre-training in Mammographycited by 9
  4. 2025Describe, Adapt and Combine: Empowering CLIP Encoders for Open-set 3D Object Retrievalcited by 5
  5. 2024PEVA-Net: Prompt-Enhanced View Aggregation Network for Zero/Few-Shot Multi-View 3D Shape Recognitioncited by 3
  6. 2025TeDA: Boosting Vision-Lanuage Models for Zero-Shot 3D Object Retrieval via Testing-time Distribution Alignmentcited by 2
  7. 2024A Dataset and Framework for Learning State-invariant Object Representationscited by 1
  8. 2025Pyramid Token Pruning for High-Resolution Large Vision-Language Models via Region, Token, and Instruction-Guided Importancecited by 1
  9. 2026Lightweight 3D Feature Pretraining by Bayesian Inversion of 2D Foundation Modelscited by 1
  10. 2026DINO Eats CLIP: Adapting Beyond Knowns for Open-set 3D Object Retrievalcited by 0
  11. 2026VLRC: Vision-Language Reprojection Consistency as a scalable signal for better feed-forward 3D pretrainingcited by 0