# Leveraging Uncertainty Estimates for Predicting Segmentation Quality

arXiv: https://arxiv.org/abs/1807.00502 · HTML: https://mlsanity.com/paper/1807.00502/

## Key facts

- Cited by 44 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 2 Jul 2018.

## Built on by (most cited first, 30 of 44)

- [Diffusion Models for Implicit Image Segmentation Ensembles](https://mlsanity.com/paper/2112.03145/) (2021)
- [Trustworthy clinical AI solutions: a unified review of uncertainty quantification in deep learning models for medical image analysis](https://mlsanity.com/paper/2210.03736/) (2022)
- [A Review of Uncertainty Estimation and its Application in Medical Imaging](https://mlsanity.com/paper/2302.08119/) (2023)
- [A review of uncertainty quantification in medical image analysis: probabilistic and non-probabilistic methods](https://mlsanity.com/paper/2310.06873/) (2023)
- [A Survey on Deep Learning for Skin Lesion Segmentation](https://mlsanity.com/paper/2206.00356/) (2022)
- [Calibrating the Dice loss to handle neural network overconfidence for biomedical image segmentation](https://mlsanity.com/paper/2111.00528/) (2021)
- [Hierarchical Deep Network with Uncertainty-aware Semi-supervised Learning for Vessel Segmentation](https://mlsanity.com/paper/2105.14732/) (2021)
- [Robust Mutual Learning for Semi-supervised Semantic Segmentation](https://mlsanity.com/paper/2106.00609/) (2021)
- [Uncertainty-aware Multi-modal Learning via Cross-modal Random Network Prediction](https://mlsanity.com/paper/2207.10851/) (2022)
- [Comparative Benchmarking of Failure Detection Methods in Medical Image Segmentation: Unveiling the Role of Confidence Aggregation](https://mlsanity.com/paper/2406.03323/) (2024)
- [Enhanced Isotropy Maximization Loss: Seamless and High-Performance Out-of-Distribution Detection Simply Replacing the SoftMax Loss](https://mlsanity.com/paper/2105.14399/) (2021)
- [Pixel-wise Gradient Uncertainty for Convolutional Neural Networks applied to Out-of-Distribution Segmentation](https://mlsanity.com/paper/2303.06920/) (2023)
- [FSNet: A Failure Detection Framework for Semantic Segmentation](https://mlsanity.com/paper/2108.08748/) (2021)
- [Building Extraction from Remote Sensing Images via an Uncertainty-Aware Network](https://mlsanity.com/paper/2307.12309/) (2023)
- [Calibration and Uncertainty for multiRater Volume Assessment in multiorgan Segmentation (CURVAS) challenge results](https://mlsanity.com/paper/2505.08685/) (2025)
- [U-LanD: Uncertainty-Driven Video Landmark Detection](https://mlsanity.com/paper/2102.01586/) (2021)
- [False Negative Reduction in Video Instance Segmentation using Uncertainty Estimates](https://mlsanity.com/paper/2106.14474/) (2021)
- [Evaluating subgroup disparity using epistemic uncertainty in mammography](https://mlsanity.com/paper/2107.02716/) (2021)
- [Improving the Reliability of Semantic Segmentation of Medical Images by Uncertainty Modeling with Bayesian Deep Networks and Curriculum Learning](https://mlsanity.com/paper/2108.11693/) (2021)
- [Informative Priors Improve the Reliability of Multimodal Clinical Data Classification](https://mlsanity.com/paper/2312.00794/) (2023)
- [A Review of Bayesian Uncertainty Quantification in Deep Probabilistic Image Segmentation](https://mlsanity.com/paper/2411.16370/) (2024)
- [Using uncertainty estimation to reduce false positives in liver lesion detection](https://mlsanity.com/paper/2101.04386/) (2021)
- [Measurement of Hybrid Rocket Solid Fuel Regression Rate for a Slab Burner using Deep Learning](https://mlsanity.com/paper/2108.11276/) (2021)
- [Deep Quantile Regression for Uncertainty Estimation in Unsupervised and Supervised Lesion Detection](https://mlsanity.com/paper/2109.09374/) (2021)
- [Interactive Medical Image Segmentation with Self-Adaptive Confidence Calibration](https://mlsanity.com/paper/2111.07716/) (2021)
- [From Registration Uncertainty to Segmentation Uncertainty](https://mlsanity.com/paper/2403.05111/) (2024)
- [A Decoupled Uncertainty Model for MRI Segmentation Quality Estimation](https://mlsanity.com/paper/2109.02413/) (2021)
- [Influence of uncertainty estimation techniques on false-positive reduction in liver lesion detection](https://mlsanity.com/paper/2206.10911/) (2022)
- [Towards Ground-truth-free Evaluation of Any Segmentation in Medical Images](https://mlsanity.com/paper/2409.14874/) (2024)
- [Combined Data and Deep Learning Model Uncertainties: An Application to the Measurement of Solid Fuel Regression Rate](https://mlsanity.com/paper/2210.14287/) (2022)

## Abstract (arXiv)

The use of deep learning for medical imaging has seen tremendous growth in the research community. One reason for the slow uptake of these systems in the clinical setting is that they are complex, opaque and tend to fail silently. Outside of the medical imaging domain, the machine learning community has recently proposed several techniques for quantifying model uncertainty (i.e.~a model knowing when it has failed). This is important in practical settings, as we can refer such cases to manual inspection or correction by humans. In this paper, we aim to bring these recent results on estimating uncertainty to bear on two important outputs in deep learning-based segmentation. The first is producing spatial uncertainty maps, from which a clinician can observe where and why a system thinks it is failing. The second is quantifying an image-level prediction of failure, which is useful for isolating specific cases and removing them from automated pipelines. We also show that reasoning about spatial uncertainty, the first output, is a useful intermediate representation for generating segmentation quality predictions, the second output. We propose a two-stage architecture for producing these measures of uncertainty, which can accommodate any deep learning-based medical segmentation pipeline.