mlsanityData to 7 Oct 2026

Confidence-based Out-of-Distribution Detection: A Comparative Study and Analysis

arXiv 2107.02568 ↗First posted 6 Jul 2021cs.CVcited by 34

Not on the map. Why?

Abstract (arXiv)

Image classification models deployed in the real world may receive inputs outside the intended data distribution. For critical applications such as clinical decision making, it is important that a model can detect such out-of-distribution (OOD) inputs and express its uncertainty. In this work, we assess the capability of various state-of-the-art approaches for confidence-based OOD detection through a comparative study and in-depth analysis. First, we leverage a computer vision benchmark to reproduce and compare multiple OOD detection methods. We then evaluate their capabilities on the challenging task of disease classification using chest X-rays. Our study shows that high performance in a computer vision task does not directly translate to accuracy in a medical imaging task. We analyse factors that affect performance of the methods between the two tasks. Our results provide useful insights for developing the next generation of OOD detection methods.

Key facts. Cited by 34 papers in the map (data through 7 Oct 2026). Builds on 21 of its 34 references that are in the map. First posted on arXiv 6 Jul 2021.

Builds on

21 of 34 references.

References · 21

  1. 2017On Calibration of Modern Neural Networkscited by 4,860
  2. 2015Dropout as a Bayesian Approximation: Representing Model Uncertainty in Deep Learningcited by 4,416
  3. 2016Simple and Scalable Predictive Uncertainty Estimation using Deep Ensemblescited by 3,888
  4. 2016Wide Residual Networkscited by 3,057
  5. 2016A Baseline for Detecting Misclassified and Out-of-Distribution Examples in Neural Networkscited by 2,362
  6. 2018A Simple Unified Framework for Detecting Out-of-Distribution Samples and Adversarial Attackscited by 1,472
  7. 2019CheXpert: A Large Chest Radiograph Dataset with Uncertainty Labels and Expert Comparisoncited by 1,410
  8. 2017Enhancing The Reliability of Out-of-distribution Image Detection in Neural Networkscited by 1,325
  9. 2018Deep Anomaly Detection with Outlier Exposurecited by 939
  10. 2017Unsupervised Anomaly Detection with Generative Adversarial Networks to Guide Marker Discoverycited by 566
  11. 2020CSI: Novelty Detection via Contrastive Learning on Distributionally Shifted Instancescited by 441
  12. 2020A Unifying Review of Deep and Shallow Anomaly Detectioncited by 423
9 more
  1. 2019Confidence Calibration and Predictive Uncertainty Estimation for Deep Medical Image Segmentationcited by 162
  2. 2018Exploring Uncertainty Measures in Deep Networks for Multiple Sclerosis Lesion Detection and Segmentationcited by 146
  3. 2020Autoencoders for Unsupervised Anomaly Segmentation in Brain MR Images: A Comparative Studycited by 146
  4. 2020Stochastic Segmentation Networks: Modelling Spatially Correlated Aleatoric Uncertaintycited by 100
  5. 2021Does Your Dermatology Classifier Know What It Doesn't Know? Detecting the Long-Tail of Unseen Conditionscited by 68
  6. 2020Unsupervised Lesion Detection via Image Restoration with a Normative Priorcited by 65
  7. 2020Detecting Outliers with Foreign Patch Interpolationcited by 44
  8. 2021Unsupervised Brain Anomaly Detection and Segmentation with Transformerscited by 34
  9. 2017Bayesian Image Quality Transfer with CNNs: Exploring Uncertainty in dMRI Super-Resolutioncited by 29

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2021: 2; 2022: 14; 2023: 6; 2024: 6; 2025: 3; 2026: 3.

34 papers.

Sort:

  1. 2022Efficient Test-Time Model Adaptation without Forgettingcited by 467
  2. 2024Test-Time Model Adaptation with Only Forward Passescited by 74
  3. 2022Trustworthy clinical AI solutions: a unified review of uncertainty quantification in deep learning models for medical image analysiscited by 62
  4. 2023A framework for benchmarking class-out-of-distribution detection and its application to ImageNetcited by 31
  5. 2024Out-of-distribution Detection in Medical Image Analysis: A surveycited by 22
  6. 2024Uncertainty-Calibrated Test-Time Model Adaptation without Forgettingcited by 20
  7. 2021CertainNet: Sampling-free Uncertainty Estimation for Object Detectioncited by 15
  8. 2023On the use of Mahalanobis distance for out-of-distribution detection with neural networks for medical imagingcited by 15
  9. 2022Towards out of distribution generalization for problems in mechanicscited by 11
  10. 2025Towards Objective Fine-tuning: How LLMs' Prior Knowledge Causes Potential Poor Calibration?cited by 11
  11. 2022Recommendations on test datasets for evaluating AI solutions in pathologycited by 8
  12. 2022Surgical Workflow Recognition: from Analysis of Challenges to Architectural Studycited by 6
22 more
  1. 2023Dimensionality Reduction for Improving Out-of-Distribution Detection in Medical Image Segmentationcited by 6
  2. 2021Test for non-negligible adverse shiftscited by 5
  3. 2022Is it all a cluster game? -- Exploring Out-of-Distribution Detection based on Clustering in the Embedding Spacecited by 5
  4. 2022Improving Uncertainty-based Out-of-Distribution Detection for Medical Image Segmentationcited by 5
  5. 2023Multi-layer Aggregation as a key to feature-based OOD detectioncited by 5
  6. 2022NovelCraft: A Dataset for Novelty Detection and Discovery in Open Worldscited by 4
  7. 2022Test Time Transform Prediction for Open Set Histopathological Image Recognitioncited by 3
  8. 2022nnOOD: A Framework for Benchmarking Self-supervised Anomaly Localisation Methodscited by 2
  9. 2023Distance Matters For Improving Performance Estimation Under Covariate Shiftcited by 2
  10. 2024Situation Monitor: Diversity-Driven Zero-Shot Out-of-Distribution Detection using Budding Ensemble Architecture for Object Detectioncited by 2
  11. 2024Evaluating Self-Supervised Learning in Medical Imaging: A Benchmark for Robustness, Generalizability, and Multi-Domain Impactcited by 2
  12. 2022Which models are innately best at uncertainty estimation?cited by 1
  13. 2022Uncertainty estimations methods for a deep learning model to aid in clinical decision-making -- a clinician's perspectivecited by 1
  14. 2023Unsupervised out-of-distribution detection for safer robotically guided retinal microsurgerycited by 1
  15. 2024DAGnosis: Localized Identification of Data Inconsistencies using Structurescited by 1
  16. 2025Tumor-anchored deep feature random forests for out-of-distribution detection in lung cancer segmentationcited by 1
  17. 2026ZOTTA: Test-Time Adaptation with Gradient-Free Zeroth-Order Optimizationcited by 1
  18. 2022Exploring How Anomalous Model Input and Output Alerts Affect Decision-Making in Healthcarecited by 0
  19. 2022Solving Sample-Level Out-of-Distribution Detection on 3D Medical Imagescited by 0
  20. 2025Influence of Classification Task and Distribution Shift Type on OOD Detection in Fetal Ultrasoundcited by 0
  21. 2026Beyond the Class Subspace: Teacher-Guided Training for Reliable Out-of-Distribution Detection in Single-Domain Modelscited by 0
  22. 2026VOLTA: The Surprising Ineffectiveness of Auxiliary Losses for Calibrated Deep Learningcited by 0