# Confidence Calibration and Predictive Uncertainty Estimation for Deep Medical Image Segmentation

arXiv: https://arxiv.org/abs/1911.13273 · HTML: https://mlsanity.com/paper/1911.13273/

## Key facts

- Cited by 163 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 29 Nov 2019.

## Built on by (most cited first, 30 of 163)

- [A Survey of Uncertainty in Deep Neural Networks](https://mlsanity.com/paper/2107.03342/) (2021)
- [SegmentMeIfYouCan: A Benchmark for Anomaly Segmentation](https://mlsanity.com/paper/2104.14812/) (2021)
- [Recent advances and clinical applications of deep learning in medical image analysis](https://mlsanity.com/paper/2105.13381/) (2021)
- [Trustworthy clinical AI solutions: a unified review of uncertainty quantification in deep learning models for medical image analysis](https://mlsanity.com/paper/2210.03736/) (2022)
- [A Review of Uncertainty Estimation and its Application in Medical Imaging](https://mlsanity.com/paper/2302.08119/) (2023)
- [Local contrastive loss with pseudo-label based self-training for semi-supervised medical image segmentation](https://mlsanity.com/paper/2112.09645/) (2021)
- [Rethinking Semi-Supervised Medical Image Segmentation: A Variance-Reduction Perspective](https://mlsanity.com/paper/2302.01735/) (2023)
- [A review of uncertainty quantification in medical image analysis: probabilistic and non-probabilistic methods](https://mlsanity.com/paper/2310.06873/) (2023)
- [A comprehensive survey on deep active learning in medical image analysis](https://mlsanity.com/paper/2310.14230/) (2023)
- [Confidence-based Out-of-Distribution Detection: A Comparative Study and Analysis](https://mlsanity.com/paper/2107.02568/) (2021)
- [TBraTS: Trusted Brain Tumor Segmentation](https://mlsanity.com/paper/2206.09309/) (2022)
- [Mine yOur owN Anatomy: Revisiting Medical Image Segmentation with Extremely Limited Labels](https://mlsanity.com/paper/2209.13476/) (2022)
- [On Calibrating Semantic Segmentation Models: Analyses and An Algorithm](https://mlsanity.com/paper/2212.12053/) (2022)
- [ValUES: A Framework for Systematic Validation of Uncertainty Estimation in Semantic Segmentation](https://mlsanity.com/paper/2401.08501/) (2024)
- [CKD-TransBTS: Clinical Knowledge-Driven Hybrid Transformer with Modality-Correlated Cross-Attention for Brain Tumor Segmentation](https://mlsanity.com/paper/2207.07370/) (2022)
- [Medical Image Segmentation with Limited Supervision: A Review of Deep Network Models](https://mlsanity.com/paper/2103.00429/) (2021)
- [Calibrating Uncertainty for Semi-Supervised Crowd Counting](https://mlsanity.com/paper/2308.09887/) (2023)
- [Out-of-distribution Detection in Medical Image Analysis: A survey](https://mlsanity.com/paper/2404.18279/) (2024)
- [Region-Based Evidential Deep Learning to Quantify Uncertainty and Improve Robustness of Brain Tumor Segmentation](https://mlsanity.com/paper/2208.06038/) (2022)
- [Detecting when pre-trained nnU-Net models fail silently for Covid-19 lung lesion segmentation](https://mlsanity.com/paper/2107.05975/) (2021)
- [A Unified Benchmark for the Unknown Detection Capability of Deep Neural Networks](https://mlsanity.com/paper/2112.00337/) (2021)
- [Efficient Bayesian Uncertainty Estimation for nnU-Net](https://mlsanity.com/paper/2212.06278/) (2022)
- [Calibrating the Dice loss to handle neural network overconfidence for biomedical image segmentation](https://mlsanity.com/paper/2111.00528/) (2021)
- [Orthogonal Ensemble Networks for Biomedical Image Segmentation](https://mlsanity.com/paper/2105.10827/) (2021)
- [Transformer-based out-of-distribution detection for clinically safe segmentation](https://mlsanity.com/paper/2205.10650/) (2022)
- [ACLS: Adaptive and Conditional Label Smoothing for Network Calibration](https://mlsanity.com/paper/2308.11911/) (2023)
- [Adaptive Learn-then-Test: Statistically Valid and Efficient Hyperparameter Selection](https://mlsanity.com/paper/2409.15844/) (2024)
- [Maximum Entropy on Erroneous Predictions (MEEP): Improving model calibration for medical image segmentation](https://mlsanity.com/paper/2112.12218/) (2021)
- [Translation Consistent Semi-supervised Segmentation for 3D Medical Images](https://mlsanity.com/paper/2203.14523/) (2022)
- [Uncertainty-aware Medical Diagnostic Phrase Identification and Grounding](https://mlsanity.com/paper/2404.06798/) (2024)

## Abstract (arXiv)

Fully convolutional neural networks (FCNs), and in particular U-Nets, have achieved state-of-the-art results in semantic segmentation for numerous medical imaging applications. Moreover, batch normalization and Dice loss have been used successfully to stabilize and accelerate training. However, these networks are poorly calibrated i.e. they tend to produce overconfident predictions both in correct and erroneous classifications, making them unreliable and hard to interpret. In this paper, we study predictive uncertainty estimation in FCNs for medical image segmentation. We make the following contributions: 1) We systematically compare cross entropy loss with Dice loss in terms of segmentation quality and uncertainty estimation of FCNs; 2) We propose model ensembling for confidence calibration of the FCNs trained with batch normalization and Dice loss; 3) We assess the ability of calibrated FCNs to predict segmentation quality of structures and detect out-of-distribution test examples. We conduct extensive experiments across three medical image segmentation applications of the brain, the heart, and the prostate to evaluate our contributions. The results of this study offer considerable insight into the predictive uncertainty estimation and out-of-distribution detection in medical image segmentation and provide practical recipes for confidence calibration. Moreover, we consistently demonstrate that model ensembling improves confidence calibration.