# Benchmarking common uncertainty estimation methods with histopathological images under domain shift and label noise

arXiv: https://arxiv.org/abs/2301.01054 · HTML: https://mlsanity.com/paper/2301.01054/

## Key facts

- Cited by 15 papers in the map (data through 14 Oct 2026).
- Builds on 24 of its 53 references that are in the map.
- First posted on arXiv 3 Jan 2023.

## Builds on (most cited first, 24 of 24)

- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [On Calibration of Modern Neural Networks](https://mlsanity.com/paper/1706.04599/) (2017)
- [Dropout as a Bayesian Approximation: Representing Model Uncertainty in Deep Learning](https://mlsanity.com/paper/1506.02142/) (2015)
- [Simple and Scalable Predictive Uncertainty Estimation using Deep Ensembles](https://mlsanity.com/paper/1612.01474/) (2016)
- [Benchmarking Neural Network Robustness to Common Corruptions and Perturbations](https://mlsanity.com/paper/1903.12261/) (2019)
- [Optuna: A Next-generation Hyperparameter Optimization Framework](https://mlsanity.com/paper/1907.10902/) (2019)
- [Variational Inference: A Review for Statisticians](https://mlsanity.com/paper/1601.00670/) (2016)
- [Can You Trust Your Model's Uncertainty? Evaluating Predictive Uncertainty Under Dataset Shift](https://mlsanity.com/paper/1906.02530/) (2019)
- [Attention-based Deep Multiple Instance Learning](https://mlsanity.com/paper/1802.04712/) (2018)
- [Selective Classification for Deep Neural Networks](https://mlsanity.com/paper/1705.08500/) (2017)
- [A Review of Uncertainty Quantification in Deep Learning: Techniques, Applications and Challenges](https://mlsanity.com/paper/2011.06225/) (2020)
- [Data Efficient and Weakly Supervised Computational Pathology on Whole Slide Images](https://mlsanity.com/paper/2004.09666/) (2020)
- [Weight Uncertainty in Neural Networks](https://mlsanity.com/paper/1505.05424/) (2015)
- [Measuring Calibration in Deep Learning](https://mlsanity.com/paper/1904.01685/) (2019)
- [Pitfalls of In-Domain Uncertainty Estimation and Ensembling in Deep Learning](https://mlsanity.com/paper/2002.06470/) (2020)
- [Quantifying the effects of data augmentation and stain color normalization in convolutional neural networks for computational pathology](https://mlsanity.com/paper/1902.06543/) (2019)
- [Deep learning with noisy labels: exploring techniques and remedies in medical image analysis](https://mlsanity.com/paper/1912.02911/) (2019)
- [Exploring Uncertainty Measures in Deep Networks for Multiple Sclerosis Lesion Detection and Segmentation](https://mlsanity.com/paper/1808.01200/) (2018)
- [Flipout: Efficient Pseudo-Independent Weight Perturbations on Mini-Batches](https://mlsanity.com/paper/1803.04386/) (2018)
- [On the Practicality of Deterministic Epistemic Uncertainty](https://mlsanity.com/paper/2107.00649/) (2021)
- [A Call to Reflect on Evaluation Practices for Failure Detection in Image Classification](https://mlsanity.com/paper/2211.15259/) (2022)
- [Benchmarking Bayesian Deep Learning on Diabetic Retinopathy Detection Tasks](https://mlsanity.com/paper/2211.12717/) (2022)
- [A Generalized Deep Learning Framework for Whole-Slide Image Segmentation and Analysis](https://mlsanity.com/paper/2001.00258/) (2020)

## Built on by (most cited first, 15 of 15)

- [Probabilistic Contrastive Learning Recovers the Correct Aleatoric Uncertainty of Ambiguous Inputs](https://mlsanity.com/paper/2302.02865/) (2023)
- [Beyond Deep Ensembles: A Large-Scale Evaluation of Bayesian Deep Learning under Distribution Shift](https://mlsanity.com/paper/2306.12306/) (2023)
- [Credal Wrapper of Model Averaging for Uncertainty Estimation in Classification](https://mlsanity.com/paper/2405.15047/) (2024)
- [MERIT: Multi-view evidential learning for reliable and interpretable liver fibrosis staging](https://mlsanity.com/paper/2405.02918/) (2024)
- [MUBen: Benchmarking the Uncertainty of Molecular Representation Models](https://mlsanity.com/paper/2306.10060/) (2023)
- [An Axiomatic Assessment of Entropy- and Variance-based Uncertainty Quantification in Regression](https://mlsanity.com/paper/2504.18433/) (2025)
- [Learning Credal Ensembles via Distributionally Robust Optimization](https://mlsanity.com/paper/2602.08470/) (2026)
- [Pitfalls of Conformal Predictions for Medical Image Classification](https://mlsanity.com/paper/2506.18162/) (2025)
- [Benchmarking Real-World Medical Image Classification with Noisy Labels: Challenges, Practice, and Outlook](https://mlsanity.com/paper/2512.09315/) (2025)
- [EDUE: Expert Disagreement-Guided One-Pass Uncertainty Estimation for Medical Image Segmentation](https://mlsanity.com/paper/2403.16594/) (2024)
- [Temporal Distribution Shift in Real-World Pharmaceutical Data: Implications for Uncertainty Quantification in QSAR Models](https://mlsanity.com/paper/2502.03982/) (2025)
- [Set-based v.s. Distribution-based Representations of Epistemic Uncertainty: A Comparative Study](https://mlsanity.com/paper/2602.22747/) (2026)
- [On the calibration of neural networks for histological slide-level classification](https://mlsanity.com/paper/2312.09719/) (2023)
- [Hallucination Detection in Virtually-Stained Histology: A Latent Space Baseline](https://mlsanity.com/paper/2411.15060/) (2024)
- [Autonomous Drift Learning in Data Streams: A Unified Perspective](https://mlsanity.com/paper/2605.01295/) (2026)

## Abstract (arXiv)

In the past years, deep learning has seen an increase in usage in the domain of histopathological applications. However, while these approaches have shown great potential, in high-risk environments deep learning models need to be able to judge their uncertainty and be able to reject inputs when there is a significant chance of misclassification. In this work, we conduct a rigorous evaluation of the most commonly used uncertainty and robustness methods for the classification of Whole Slide Images, with a focus on the task of selective classification, where the model should reject the classification in situations in which it is uncertain. We conduct our experiments on tile-level under the aspects of domain shift and label noise, as well as on slide-level. In our experiments, we compare Deep Ensembles, Monte-Carlo Dropout, Stochastic Variational Inference, Test-Time Data Augmentation as well as ensembles of the latter approaches. We observe that ensembles of methods generally lead to better uncertainty estimates as well as an increased robustness towards domain shifts and label noise, while contrary to results from classical computer vision benchmarks no systematic gain of the other methods can be shown. Across methods, a rejection of the most uncertain samples reliably leads to a significant increase in classification accuracy on both in-distribution as well as out-of-distribution data. Furthermore, we conduct experiments comparing these methods under varying conditions of label noise. Lastly, we publish our code framework to facilitate further research on uncertainty estimation on histopathological data.