mlsanityData to 7 Oct 2026

Exploring the Limits of Out-of-Distribution Detection

arXiv 2106.03004 ↗First posted 6 Jun 2021cs.LGcited by 283

Not on the map. Why?

Cited by 23 in Multimodal LLMs / vision-language models, but placed by its own abstract, which didn’t fit.

Abstract (arXiv)

Near out-of-distribution detection (OOD) is a major challenge for deep neural networks. We demonstrate that large-scale pre-trained transformers can significantly improve the state-of-the-art (SOTA) on a range of near OOD tasks across different data modalities. For instance, on CIFAR-100 vs CIFAR-10 OOD detection, we improve the AUROC from 85% (current SOTA) to more than 96% using Vision Transformers pre-trained on ImageNet-21k. On a challenging genomics OOD detection benchmark, we improve the AUROC from 66% to 77% using transformers and unsupervised pre-training. To further improve performance, we explore the few-shot outlier exposure setting where a few examples from outlier classes may be available; we show that pre-trained transformers are particularly well-suited for outlier exposure, and that the AUROC of OOD detection on CIFAR-100 vs CIFAR-10 can be improved to 98.7% with just 1 image per OOD class, and 99.46% with 10 images per OOD class. For multi-modal image-text pre-trained transformers such as CLIP, we explore a new way of using just the names of outlier classes as a sole source of information without any accompanying images, and show that this outperforms previous SOTA on standard vision OOD benchmark tasks.

Key facts. Cited by 283 papers in the map (data through 7 Oct 2026). Builds on 40 of its 58 references that are in the map. First posted on arXiv 6 Jun 2021.

Builds on

40 of 58 references.

References · 40

  1. 2017Attention Is All You Needcited by 59,932
  2. 2015Deep Residual Learning for Image Recognitioncited by 51,809
  3. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,204
  4. 2014Adam: A Method for Stochastic Optimizationcited by 36,960
  5. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,456
  6. 2020Language Models are Few-Shot Learnerscited by 32,199
  7. 2020An Image is Worth 16x16 Words: Transformers for Image Recognition at Scalecited by 29,915
  8. 2019Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformercited by 14,491
  9. 2021Emerging Properties in Self-Supervised Vision Transformerscited by 6,893
  10. 2017On Calibration of Modern Neural Networkscited by 4,860
  11. 2016Simple and Scalable Predictive Uncertainty Estimation using Deep Ensemblescited by 3,888
  12. 2021Switch Transformers: Scaling to Trillion Parameter Models with Simple and Efficient Sparsitycited by 3,014
28 more
  1. 2016A Baseline for Detecting Misclassified and Out-of-Distribution Examples in Neural Networkscited by 2,362
  2. 2018Deep contextualized word representationscited by 2,285
  3. 2013Describing Textures in the Wildcited by 2,110
  4. 2020Shortcut Learning in Deep Neural Networkscited by 1,928
  5. 2016Concrete Problems in AI Safetycited by 1,582
  6. 2018A Simple Unified Framework for Detecting Out-of-Distribution Samples and Adversarial Attackscited by 1,472
  7. 2021MLP-Mixer: An all-MLP Architecture for Visioncited by 1,441
  8. 2017Enhancing The Reliability of Out-of-distribution Image Detection in Neural Networkscited by 1,325
  9. 2018Deep Anomaly Detection with Outlier Exposurecited by 939
  10. 2014Deep Neural Networks are Easily Fooled: High Confidence Predictions for Unrecognizable Imagescited by 774
  11. 2019Big Transfer (BiT): General Visual Representation Learningcited by 766
  12. 2021ImageNet-21K Pretraining for the Massescited by 607
  13. 2019Using Self-Supervised Learning Can Improve Model Robustness and Uncertaintycited by 511
  14. 2019Likelihood Ratios for Out-of-Distribution Detectioncited by 451
  15. 2021Intriguing Properties of Vision Transformerscited by 446
  16. 2020Simple and Principled Uncertainty Estimation with Deterministic Deep Learning via Distance Awarenesscited by 389
  17. 2019Using Pre-Training Can Improve Model Robustness and Uncertaintycited by 373
  18. 2020Pretrained Transformers Improve Out-of-Distribution Robustnesscited by 293
  19. 2021Understanding Robustness of Transformers for Image Classificationcited by 250
  20. 2021Vision Transformers are Robust Learnerscited by 208
  21. 2015Semi-supervised Sequence Learningcited by 198
  22. 2020Contrastive Training for Improved Out-of-Distribution Detectioncited by 186
  23. 2021On the Robustness of Vision Transformers to Adversarial Examplescited by 167
  24. 2020ProtTrans: Towards Cracking the Language of Life's Code Through Self-Supervised Deep Learning and High Performance Computingcited by 162
  25. 2021Towards Robust Vision Transformercited by 137
  26. 2020Hybrid Models for Open Set Recognitioncited by 109
  27. 2021Does Your Dermatology Classifier Know What It Doesn't Know? Detecting the Long-Tail of Unseen Conditionscited by 68
  28. 2019Hybrid Models with Deep and Invertible Featurescited by 41

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2021: 11; 2022: 50; 2023: 84; 2024: 69; 2025: 50; 2026: 19.

283 papers.

Sort:

By area:

  1. 2022Training a Helpful and Harmless Assistant with Reinforcement Learning from Human Feedbackcited by 3,634
  2. 2021Generalized Out-of-Distribution Detection: A Surveycited by 796
  3. 2023Scaling Vision Transformers to 22 Billion Parameterscited by 690
  4. 2022ViM: Out-Of-Distribution with Virtual-logit Matchingcited by 404
  5. 2021Open-Set Recognition: a Good Closed-Set Classifier is All You Need?cited by 351
  6. 2023WinCLIP: Zero-/Few-Shot Anomaly Classification and Segmentationcited by 311
  7. 2022Transformers in Medical Imaging: A Surveycited by 245
  8. 2022Delving into Out-of-Distribution Detection with Vision-Language Representationscited by 215
  9. 2023Learning to Exploit Temporal Structure for Biomedical Vision-Language Processingcited by 210
  10. 2021A Simple Fix to Mahalanobis Distance for Improving Near-OOD Detectioncited by 206
  11. 2021A Unified Survey on Anomaly, Novelty, Open-Set, and Out-of-Distribution Detection: Solutions and Future Challengescited by 154
  12. 2021Zero-Shot Out-of-Distribution Detection Based on the Pre-trained Model CLIPcited by 151
38 more
  1. 2022Reconstructing Training Data with Informed Adversariescited by 143
  2. 2023In or Out? Fixing ImageNet Out-of-Distribution Detection Evaluationcited by 132
  3. 2023CLIPN for Zero-Shot OOD Detection: Teaching CLIP to Say Nocited by 130
  4. 2023Non-Parametric Outlier Synthesiscited by 129
  5. 2022Goal Misgeneralization: Why Correct Specifications Aren't Enough For Correct Goalscited by 120
  6. 2022Is Out-of-Distribution Detection Learnable?cited by 109
  7. 2022POEM: Out-of-Distribution Detection with Posterior Samplingcited by 108
  8. 2022Multimodal Masked Autoencoders Learn Transferable Representationscited by 107
  9. 2023LoCoOp: Few-Shot Out-of-Distribution Detection via Prompt Learningcited by 107
  10. 2022Plex: Towards Reliability using Pretrained Large Model Extensionscited by 102
  11. 2022Survey on Federated Learning Threats: concepts, taxonomy on attacks and defences, experimental study and challengescited by 81
  12. 2023Dream the Impossible: Outlier Imagination with Diffusion Modelscited by 77
  13. 2021Prior and Posterior Networks: A Survey on Evidential Deep Learning Methods For Uncertainty Estimationcited by 71
  14. 2024Negative Label Guided OOD Detection with Pretrained Vision-Language Modelscited by 66
  15. 2023Nearest Neighbor Guidance for Out-of-Distribution Detectioncited by 64
  16. 2022RankFeat: Rank-1 Feature Removal for Out-of-distribution Detectioncited by 52
  17. 2022A Call to Reflect on Evaluation Practices for Failure Detection in Image Classificationcited by 51
  18. 2023Fast Decision Boundary based Out-of-Distribution Detectorcited by 48
  19. 2023Rethinking Out-of-distribution (OOD) Detection: Masked Image Modeling is All You Needcited by 46
  20. 2022Partial and Asymmetric Contrastive Learning for Out-of-Distribution Detection in Long-Tailed Recognitioncited by 44
  21. 2022Isolation and Impartial Aggregation: A Paradigm of Incremental Learning without Interferencecited by 44
  22. 2024A Closer Look at the Robustness of Contrastive Language-Image Pre-Training (CLIP)cited by 44
  23. 2023ID-like Prompt Learning for Few-Shot Out-of-Distribution Detectioncited by 42
  24. 2023Learning Generalized Zero-Shot Learners for Open-Domain Image Geolocalizationcited by 41
  25. 2023NECO: NEural Collapse Based Out-of-distribution detectioncited by 41
  26. 2023OpenMix: Exploring Outlier Samples for Misclassification Detectioncited by 38
  27. 2023How Does Fine-Tuning Impact Out-of-Distribution Detection for Vision-Language Models?cited by 38
  28. 2023Diversified Outlier Exposure for Out-of-Distribution Detection via Informative Extrapolationcited by 38
  29. 2023GL-MCM: Global and Local Maximum Concept Matching for Zero-Shot Out-of-Distribution Detectioncited by 36
  30. 2024How Does Unlabeled Data Provably Help Out-of-Distribution Detection?cited by 36
  31. 2024Generalized Out-of-Distribution Detection and Beyond in Vision Language Model Era: A Surveycited by 36
  32. 2022Exposing Outlier Exposure: What Can Be Learned From Few, One, and Zero Outlier Imagescited by 35
  33. 2022An Impartial Take to the CNN vs Transformer Robustness Contestcited by 34
  34. 2023CiCo: Domain-Aware Sign Language Retrieval via Cross-Lingual Contrastive Learningcited by 32
  35. 2023A framework for benchmarking class-out-of-distribution detection and its application to ImageNetcited by 31
  36. 2024Envisioning Outlier Exposure by Large Language Models for Out-of-Distribution Detectioncited by 30
  37. 2023Neuron Activation Coverage: Rethinking Out-of-distribution Detection and Generalizationcited by 28
  38. 2022Augmenting Softmax Information for Selective Classification with Out-of-Distribution Datacited by 27

50 most cited of 283 All 283 (JSON)