mlsanityData to 7 Oct 2026

Combining pre-trained Vision Transformers and CIDER for Out Of Domain Detection

arXiv 2309.03047 ↗First posted 6 Sep 2023cs.CV cs.AIcited by 0

Not on the map. Why?

Abstract (arXiv)

Out-of-domain (OOD) detection is a crucial component in industrial applications as it helps identify when a model encounters inputs that are outside the training distribution. Most industrial pipelines rely on pre-trained models for downstream tasks such as CNN or Vision Transformers. This paper investigates the performance of those models on the task of out-of-domain detection. Our experiments demonstrate that pre-trained transformers models achieve higher detection performance out of the box. Furthermore, we show that pre-trained ViT and CNNs can be combined with refinement methods such as CIDER to improve their OOD detection performance even more. Our results suggest that transformers are a promising approach for OOD detection and set a stronger baseline for this task in many contexts

Key facts. No citing papers in the map yet (data through 7 Oct 2026). Builds on 21 of its 22 references that are in the map. First posted on arXiv 6 Sep 2023.

Builds on

21 of 22 references.

References · 21

  1. 2017Attention Is All You Needcited by 59,932
  2. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,204
  3. 2020Language Models are Few-Shot Learnerscited by 32,199
  4. 2020An Image is Worth 16x16 Words: Transformers for Image Recognition at Scalecited by 29,915
  5. 2016A Baseline for Detecting Misclassified and Out-of-Distribution Examples in Neural Networkscited by 2,362
  6. 2021BEiT: BERT Pre-Training of Image Transformerscited by 2,360
  7. 2018A Simple Unified Framework for Detecting Out-of-Distribution Samples and Adversarial Attackscited by 1,472
  8. 2017Enhancing The Reliability of Out-of-distribution Image Detection in Neural Networkscited by 1,325
  9. 2020Energy-based Out-of-distribution Detectioncited by 1,296
  10. 2020A Review of Uncertainty Quantification in Deep Learning: Techniques, Applications and Challengescited by 813
  11. 2014Deep Neural Networks are Easily Fooled: High Confidence Predictions for Unrecognizable Imagescited by 774
  12. 2015Towards Open Set Deep Networkscited by 533
9 more
  1. 2019Scaling Out-of-Distribution Detection for Real-World Settingscited by 465
  2. 2022VOS: Learning What You Don't Know by Virtual Outlier Synthesiscited by 300
  3. 2022OpenOOD: Benchmarking Generalized Out-of-Distribution Detectioncited by 267
  4. 2021SSD: A Unified Framework for Self-Supervised Outlier Detectioncited by 264
  5. 2022How to Exploit Hyperspherical Embeddings for Out-of-Distribution Detection?cited by 128
  6. 2015The Extreme Value Machinecited by 78
  7. 2021Does Your Dermatology Classifier Know What It Doesn't Know? Detecting the Long-Tail of Unseen Conditionscited by 68
  8. 2021MACEst: The reliable and trustworthy Model Agnostic Confidence Estimatorcited by 3
  9. 2022Identifying Out-of-Distribution Samples in Real-Time for Safety-Critical 2D Object Detection with Margin Entropy Losscited by 3

Built on by

Not cited yet.