mlsanityData to 7 Oct 2026

Zero-Shot Out-of-Distribution Detection Based on the Pre-trained Model CLIP

arXiv 2109.02748 ↗First posted 6 Sep 2021cs.CV cs.LGcited by 151

Not on the map. Why?

Cited by 43 in Multimodal LLMs / vision-language models, but placed by its own abstract, which didn’t fit.

Abstract (arXiv)

In an out-of-distribution (OOD) detection problem, samples of known classes(also called in-distribution classes) are used to train a special classifier. In testing, the classifier can (1) classify the test samples of known classes to their respective classes and also (2) detect samples that do not belong to any of the known classes (i.e., they belong to some unknown or OOD classes). This paper studies the problem of zero-shot out-of-distribution(OOD) detection, which still performs the same two tasks in testing but has no training except using the given known class names. This paper proposes a novel yet simple method (called ZOC) to solve the problem. ZOC builds on top of the recent advances in zero-shot classification through multi-modal representation learning. It first extends the pre-trained language-vision model CLIP by training a text-based image description generator on top of CLIP. In testing, it uses the extended model to generate candidate unknown class names for each test sample and computes a confidence score based on both the known class names and candidate unknown class names for zero-shot OOD detection. Experimental results on 5 benchmark datasets for OOD detection demonstrate that ZOC outperforms the baselines by a large margin.

Key facts. Cited by 151 papers in the map (data through 7 Oct 2026). Builds on 29 of its 41 references that are in the map. First posted on arXiv 6 Sep 2021.

Builds on

29 of 41 references.

References · 29

  1. 2017Attention Is All You Needcited by 59,932
  2. 2015Deep Residual Learning for Image Recognitioncited by 51,809
  3. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,204
  4. 2014Adam: A Method for Stochastic Optimizationcited by 36,960
  5. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,456
  6. 2020An Image is Worth 16x16 Words: Transformers for Image Recognition at Scalecited by 29,915
  7. 2014Microsoft COCO: Common Objects in Contextcited by 17,498
  8. 2019The Curious Case of Neural Text Degenerationcited by 2,881
  9. 2016A Baseline for Detecting Misclassified and Out-of-Distribution Examples in Neural Networkscited by 2,362
  10. 2018A Simple Unified Framework for Detecting Out-of-Distribution Samples and Adversarial Attackscited by 1,472
  11. 2017Enhancing The Reliability of Out-of-distribution Image Detection in Neural Networkscited by 1,325
  12. 2018Hierarchical Neural Story Generationcited by 1,182
17 more
  1. 2018Deep Anomaly Detection with Outlier Exposurecited by 939
  2. 2015Scheduled Sampling for Sequence Prediction with Recurrent Neural Networkscited by 779
  3. 2016Self-critical Sequence Training for Image Captioningcited by 544
  4. 2015Towards Open Set Deep Networkscited by 533
  5. 2020CSI: Novelty Detection via Contrastive Learning on Distributionally Shifted Instancescited by 441
  6. 2020Generalized ODIN: Detecting Out-of-distribution Image without Learning from Out-of-distribution Datacited by 408
  7. 2017Training Confidence-calibrated Classifiers for Detecting Out-of-Distribution Samplescited by 389
  8. 2020Pretrained Transformers Improve Out-of-Distribution Robustnesscited by 293
  9. 2021Exploring the Limits of Out-of-Distribution Detectioncited by 283
  10. 2016Professor Forcing: A New Algorithm for Training Recurrent Networkscited by 214
  11. 2020Contrastive Training for Improved Out-of-Distribution Detectioncited by 186
  12. 2017Generative OpenMax for Multi-Class Open Set Classificationcited by 157
  13. 2019C2AE: Class Conditioned Auto-Encoder for Open-set Recognitioncited by 142
  14. 2017DOC: Deep Open Classification of Text Documentscited by 109
  15. 2018Open-world Learning and Application to Product Classificationcited by 39
  16. 2018Unseen Class Discovery in Open-world Classificationcited by 20
  17. 2017Memory Based Online Learning of Deep Representations from Video Streamscited by 6

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2021: 2; 2022: 6; 2023: 36; 2024: 51; 2025: 36; 2026: 20.

151 papers.

Sort:

By area:

  1. 2021Generalized Out-of-Distribution Detection: A Surveycited by 796
  2. 2023AnomalyCLIP: Object-agnostic Prompt Learning for Zero-shot Anomaly Detectioncited by 270
  3. 2023Learning to Exploit Temporal Structure for Biomedical Vision-Language Processingcited by 210
  4. 2022ZegCLIP: Towards Adapting CLIP for Zero-shot Semantic Segmentationcited by 178
  5. 2022CREPE: Can Vision-Language Foundation Models Reason Compositionally?cited by 177
  6. 2023Raising the Bar of AI-generated Image Detection with CLIPcited by 163
  7. 2023CLIPN for Zero-Shot OOD Detection: Teaching CLIP to Say Nocited by 130
  8. 2023LoCoOp: Few-Shot Out-of-Distribution Detection via Prompt Learningcited by 107
  9. 2022A Theoretical Study on Solving Continual Learningcited by 78
  10. 2023Dream the Impossible: Outlier Imagination with Diffusion Modelscited by 77
  11. 2024Negative Label Guided OOD Detection with Pretrained Vision-Language Modelscited by 66
  12. 2024FiLo: Zero-Shot Anomaly Detection by Fine-Grained Description and High-Quality Localizationcited by 51
38 more
  1. 2024Learning Transferable Negative Prompts for Out-of-Distribution Detectioncited by 43
  2. 2023ID-like Prompt Learning for Few-Shot Out-of-Distribution Detectioncited by 42
  3. 2023GL-MCM: Global and Local Maximum Concept Matching for Zero-Shot Out-of-Distribution Detectioncited by 36
  4. 2024Knowledge-Enhanced Dual-stream Zero-shot Composed Image Retrievalcited by 36
  5. 2024Generalized Out-of-Distribution Detection and Beyond in Vision Language Model Era: A Surveycited by 36
  6. 2023CLIP-S4: Language-Guided Self-Supervised Semantic Segmentationcited by 34
  7. 2023A framework for benchmarking class-out-of-distribution detection and its application to ImageNetcited by 31
  8. 2024Envisioning Outlier Exposure by Large Language Models for Out-of-Distribution Detectioncited by 30
  9. 2024PointAD: Comprehending 3D Anomalies from Points and Pixels for Zero-shot 3D Anomaly Detectioncited by 29
  10. 2024PathGen-1.6M: 1.6 Million Pathology Image-text Pairs Generation through Multi-agent Collaborationcited by 28
  11. 2024Large Language Models for Anomaly and Out-of-Distribution Detection: A Surveycited by 24
  12. 2024Out-of-Distribution Detection: A Task-Oriented Survey of Recent Advancescited by 24
  13. 2024Out-of-distribution Detection in Medical Image Analysis: A surveycited by 22
  14. 2023Building an Open-Vocabulary Video CLIP Model with Better Architectures, Optimization and Datacited by 21
  15. 2024LAPT: Label-driven Automated Prompt Tuning for OOD Detection with Vision-Language Modelscited by 21
  16. 2023SGDiff: A Style Guided Diffusion Model for Fashion Synthesiscited by 20
  17. 2024T-VSL: Text-Guided Visual Sound Source Localization in Mixturescited by 19
  18. 2023AutoLabel: CLIP-based framework for Open-set Video Domain Adaptationcited by 18
  19. 2024Overcoming the Pitfalls of Vision-Language Model Finetuning for OOD Generalizationcited by 17
  20. 2024AdaNeg: Adaptive Negative Proxy Guided OOD Detection with Vision-Language Modelscited by 16
  21. 2023Unsupervised Layer-wise Score Aggregation for Textual OOD Detectioncited by 15
  22. 2023UKnow: A Unified Knowledge Protocol with Multimodal Knowledge Graph Datasets for Reasoning and Vision-Language Pre-Trainingcited by 14
  23. 2024Local-Prompt: Extensible Local Prompts for Few-Shot Out-of-Distribution Detectioncited by 14
  24. 2024CLIPScope: Enhancing Zero-Shot OOD Detection with Bayesian Scoringcited by 13
  25. 2024Conjugated Semantic Pool Improves OOD Detection with Pre-trained Vision-Language Modelscited by 13
  26. 2023Exploring Large Language Models for Multi-Modal Out-of-Distribution Detectioncited by 12
  27. 2023FD-Align: Feature Discrimination Alignment for Fine-tuning Pre-Trained Models in Few-Shot Learningcited by 12
  28. 2023On the Powerfulness of Textual Outlier Exposure for Visual OoD Detectioncited by 12
  29. 2025Locality-Aware Zero-Shot Human-Object Interaction Detectioncited by 12
  30. 2023Three Towers: Flexible Contrastive Learning with Pretrained Image Modelscited by 11
  31. 2023Learning Adversarial Semantic Embeddings for Zero-Shot Recognition in Open Worldscited by 11
  32. 2023Spectral Prompt Tuning:Unveiling Unseen Classes for Zero-Shot Semantic Segmentationcited by 11
  33. 2023Universal Domain Adaptation from Foundation Models: A Baseline Studycited by 10
  34. 2023LMC: Large Model Collaboration with Cross-assessment for Training-Free Open-Set Object Recognitioncited by 10
  35. 2024FodFoM: Fake Outlier Data by Foundation Models Creates Stronger Visual Out-of-Distribution Detectorcited by 10
  36. 2025BBQ-V: Benchmarking Visual Stereotype Bias in Large Multimodal Modelscited by 10
  37. 2022Zero-shot Image Captioning by Anchor-augmented Vision-Language Space Alignmentcited by 8
  38. 2023Calibrated Out-of-Distribution Detection with a Generic Representationcited by 8

50 most cited of 151 All 151 (JSON)