# Lost in the Folds: When Cross-Validation Is Not a Deep Ensemble for Uncertainty Estimation

arXiv: https://arxiv.org/abs/2605.18329 · HTML: https://mlsanity.com/paper/2605.18329/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Builds on 12 of its 28 references that are in the map.
- First posted on arXiv 18 May 2026.

## Builds on (most cited first, 12 of 12)

- [Simple and Scalable Predictive Uncertainty Estimation using Deep Ensembles](https://mlsanity.com/paper/1612.01474/) (2016)
- [What Uncertainties Do We Need in Bayesian Deep Learning for Computer Vision?](https://mlsanity.com/paper/1703.04977/) (2017)
- [Can You Trust Your Model's Uncertainty? Evaluating Predictive Uncertainty Under Dataset Shift](https://mlsanity.com/paper/1906.02530/) (2019)
- [nnU-Net Revisited: A Call for Rigorous Validation in 3D Medical Image Segmentation](https://mlsanity.com/paper/2404.09556/) (2024)
- [Confidence Calibration and Predictive Uncertainty Estimation for Deep Medical Image Segmentation](https://mlsanity.com/paper/1911.13273/) (2019)
- [Assessing Reliability and Challenges of Uncertainty Estimations for Medical Image Segmentation](https://mlsanity.com/paper/1907.03338/) (2019)
- [A review of uncertainty quantification in medical image analysis: probabilistic and non-probabilistic methods](https://mlsanity.com/paper/2310.06873/) (2023)
- [ValUES: A Framework for Systematic Validation of Uncertainty Estimation in Semantic Segmentation](https://mlsanity.com/paper/2401.08501/) (2024)
- [Efficient Bayesian Uncertainty Estimation for nnU-Net](https://mlsanity.com/paper/2212.06278/) (2022)
- [Comparative Benchmarking of Failure Detection Methods in Medical Image Segmentation: Unveiling the Role of Confidence Aggregation](https://mlsanity.com/paper/2406.03323/) (2024)
- [Spatially-Aware Evaluation of Segmentation Uncertainty](https://mlsanity.com/paper/2506.16589/) (2025)
- [Uncertainty-guided annotation enhances segmentation with the human-in-the-loop](https://mlsanity.com/paper/2404.07208/) (2024)

## Built on by (most cited first, 1 of 1)

- [Assessing nnU-Net Generalization across Brain Tumor Populations in BraTS-GoAT 2026](https://mlsanity.com/paper/2609.15524/) (2026)

## Abstract (arXiv)

Ensemble disagreement is widely used as a proxy for epistemic uncertainty in medical image segmentation. In practice, many studies form ensembles via K-fold cross-validation (CV), yet refer to them as ``deep ensembles'' (DE). Because CV members are trained on different data subsets, their disagreement mixes seed-driven variability with data-exposure effects, which can change how uncertainty should be interpreted. We audit recent segmentation uncertainty studies and find that terminology--implementation mismatches are common. We then compare a standard 5-fold CV ensemble to a 5-member DE (fixed training set, different random seeds) under otherwise identical configurations on three multi-rater segmentation datasets spanning three modalities. We evaluate uncertainty for calibration, failure detection, ambiguity modeling, and robustness under distribution shift. DE match segmentation accuracy while improving calibration and failure detection, whereas CV ensembles sometimes correlate more strongly with inter-rater variability on the studied datasets. Thus, ensemble construction should be chosen to match the research question: DE for reliability-oriented use (e.g., selective referral/failure detection) and CV ensembles as a proxy for ambiguity. We provide a lightweight nnU-Net modification enabling DE training within the default pipeline.