mlsanityData to 7 Oct 2026

Out-of-Distribution Detection for Medical Applications: Guidelines for Practical Evaluation

arXiv 2109.14885 ↗First posted 30 Sep 2021cs.LGcited by 15

Not on the map. Why?

Abstract (arXiv)

Detection of Out-of-Distribution (OOD) samples in real time is a crucial safety check for deployment of machine learning models in the medical field. Despite a growing number of uncertainty quantification techniques, there is a lack of evaluation guidelines on how to select OOD detection methods in practice. This gap impedes implementation of OOD detection methods for real-world applications. Here, we propose a series of practical considerations and tests to choose the best OOD detector for a specific medical dataset. These guidelines are illustrated on a real-life use case of Electronic Health Records (EHR). Our results can serve as a guide for implementation of OOD detection methods in clinical practice, mitigating risks associated with the use of machine learning models in healthcare.

Key facts. Cited by 15 papers in the map (data through 7 Oct 2026). Builds on 27 of its 49 references that are in the map. First posted on arXiv 30 Sep 2021.

Builds on

27 of 49 references.

References · 27

  1. 2017A Unified Approach to Interpreting Model Predictionscited by 6,488
  2. 2017On Calibration of Modern Neural Networkscited by 4,860
  3. 2015Dropout as a Bayesian Approximation: Representing Model Uncertainty in Deep Learningcited by 4,416
  4. 2016Simple and Scalable Predictive Uncertainty Estimation using Deep Ensemblescited by 3,888
  5. 2015Variational Inference with Normalizing Flowscited by 2,161
  6. 2019Can You Trust Your Model's Uncertainty? Evaluating Predictive Uncertainty Under Dataset Shiftcited by 1,274
  7. 2015Weight Uncertainty in Neural Networkscited by 554
  8. 2019Scaling Out-of-Distribution Detection for Real-World Settingscited by 465
  9. 2018Do Deep Generative Models Know What They Don't Know?cited by 443
  10. 2020Simple and Principled Uncertainty Estimation with Deterministic Deep Learning via Distance Awarenesscited by 389
  11. 2018Why ReLU networks yield high-confidence predictions far away from the training data and how to mitigate the problemcited by 323
  12. 2020Why Normalizing Flows Fail to Detect Out-of-Distribution Datacited by 220
15 more
  1. 2019Explaining individual predictions when features are dependent: More accurate approximations to Shapley valuescited by 205
  2. 2020Contrastive Training for Improved Out-of-Distribution Detectioncited by 186
  3. 2018Handling Incomplete Heterogeneous Data using VAEscited by 148
  4. 2015Neural Variational Inference for Text Processingcited by 111
  5. 2021Understanding Failures in Out-of-Distribution Detection with Deep Generative Modelscited by 89
  6. 2019Discrete Flows: Invertible Generative Models of Discrete Datacited by 68
  7. 2020VAEM: a Deep Generative Model for Heterogeneous Mixed Type Datacited by 66
  8. 2020Trust Issues: Uncertainty Estimation Does Not Enable Reliable OOD Detection On Medical Tabular Datacited by 52
  9. 2020Know Your Limits: Uncertainty Estimation with ReLU Classifiers Fails at Reliable OOD Detectioncited by 26
  10. 2018Improving Explorability in Variational Inference with Annealed Variational Objectivescited by 24
  11. 2020Detecting Out-of-distribution Samples via Variational Auto-encoder with Reliable Uncertainty Estimationcited by 24
  12. 2019Tutorial: Safe and Reliable Machine Learningcited by 22
  13. 2019Nonparametric Density Estimation for High-Dimensional Data - Algorithms and Applicationscited by 18
  14. 2020Learning Discrete Distributions by Dequantizationcited by 17
  15. 2021Practical Evaluation of Out-of-Distribution Detection Methods for Image Classificationcited by 10

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2022: 2; 2023: 5; 2024: 1; 2025: 4; 2026: 3.

15 papers.

Sort:

  1. 2023Unmasking the Chameleons: A Benchmark for Out-of-Distribution Detection in Medical Tabular Datacited by 8
  2. 2023Beyond AUROC & co. for evaluating out-of-distribution detection performancecited by 7
  3. 2023A graph neural network-based model with Out-of-Distribution Robustness for enhancing Antiretroviral Therapy Outcome Prediction for HIV-1cited by 7
  4. 2022Looking for Out-of-Distribution Environments in Multi-center Critical Care Datacited by 3
  5. 2023Model Monitoring and Robustness of In-Use Machine Learning Models: Quantifying Data Distribution Shifts Using Population Stability Indexcited by 3
  6. 2024NCDD: Nearest Centroid Distance Deficit for Out-Of-Distribution Detection in Gastrointestinal Visioncited by 3
  7. 2022Towards Robust Deep Learning using Entropic Lossescited by 0
  8. 2023Revamping AI Models in Dermatology: Overcoming Critical Challenges for Enhanced Skin Lesion Diagnosiscited by 0
  9. 2025Reliably Detecting Model Failures in Deployment Without Labelscited by 0
  10. 2025DynaSubVAE: Adaptive Subgrouping for Scalable and Robust OOD Detectioncited by 0
  11. 2025Influence of Classification Task and Distribution Shift Type on OOD Detection in Fetal Ultrasoundcited by 0
  12. 2025General OOD Detection via Model-aware and Subspace-aware Variable Prioritycited by 0
3 more
  1. 2026The Invisible Gorilla Effect in Out-of-distribution Detectioncited by 0
  2. 2026Rethinking external validation for the target population: Capturing patient-level similarity with a generative modelcited by 0
  3. 2026Training-Free Out-of-Distribution Detection for Pathology Whole-Slide Imagescited by 0