# Supervised Uncertainty Quantification for Segmentation with Multiple Annotations

arXiv: https://arxiv.org/abs/1907.01949 · HTML: https://mlsanity.com/paper/1907.01949/

## Key facts

- Cited by 42 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 3 Jul 2019.

## Built on by (most cited first, 30 of 42)

- [A Survey of Uncertainty in Deep Neural Networks](https://mlsanity.com/paper/2107.03342/) (2021)
- [Deep Unsupervised Domain Adaptation: A Review of Recent Advances and Perspectives](https://mlsanity.com/paper/2208.07422/) (2022)
- [Trustworthy clinical AI solutions: a unified review of uncertainty quantification in deep learning models for medical image analysis](https://mlsanity.com/paper/2210.03736/) (2022)
- [A Review of Uncertainty Estimation and its Application in Medical Imaging](https://mlsanity.com/paper/2302.08119/) (2023)
- [A review of uncertainty quantification in medical image analysis: probabilistic and non-probabilistic methods](https://mlsanity.com/paper/2310.06873/) (2023)
- [Stochastic Segmentation with Conditional Categorical Diffusion Models](https://mlsanity.com/paper/2303.08888/) (2023)
- [ValUES: A Framework for Systematic Validation of Uncertainty Estimation in Semantic Segmentation](https://mlsanity.com/paper/2401.08501/) (2024)
- [ACT: Semi-supervised Domain-adaptive Medical Image Segmentation with Asymmetric Co-training](https://mlsanity.com/paper/2206.02288/) (2022)
- [Generative Self-training for Cross-domain Unsupervised Tagged-to-Cine MRI Synthesis](https://mlsanity.com/paper/2106.12499/) (2021)
- [Using Soft Labels to Model Uncertainty in Medical Image Segmentation](https://mlsanity.com/paper/2109.12622/) (2021)
- [Robustness via Uncertainty-aware Cycle Consistency](https://mlsanity.com/paper/2110.12467/) (2021)
- [Modeling Multimodal Aleatoric Uncertainty in Segmentation with Mixture of Stochastic Experts](https://mlsanity.com/paper/2212.07328/) (2022)
- [Modeling Annotator Preference and Stochastic Annotation Error for Medical Image Segmentation](https://mlsanity.com/paper/2111.13410/) (2021)
- [Generalized Probabilistic U-Net for medical image segementation](https://mlsanity.com/paper/2207.12872/) (2022)
- [Effect of latent space distribution on the segmentation of images with multiple annotations](https://mlsanity.com/paper/2304.13476/) (2023)
- [Uncertainty-Guided Lung Nodule Segmentation with Feature-Aware Attention](https://mlsanity.com/paper/2110.12372/) (2021)
- [Transmission-Guided Bayesian Generative Model for Smoke Segmentation](https://mlsanity.com/paper/2303.00900/) (2023)
- [A Review of Bayesian Uncertainty Quantification in Deep Probabilistic Image Segmentation](https://mlsanity.com/paper/2411.16370/) (2024)
- [Deep Quantile Regression for Uncertainty Estimation in Unsupervised and Supervised Lesion Detection](https://mlsanity.com/paper/2109.09374/) (2021)
- [Anatomical and Diagnostic Bayesian Segmentation in Prostate MRI -Should Different Clinical Objectives Mandate Different Loss Functions?](https://mlsanity.com/paper/2110.12889/) (2021)
- [Variational Inference for Quantifying Inter-observer Variability in Segmentation of Anatomical Structures](https://mlsanity.com/paper/2201.07106/) (2022)
- [Probabilistic 3D segmentation for aleatoric uncertainty quantification in full 3D medical data](https://mlsanity.com/paper/2305.00950/) (2023)
- [Confidence Contours: Uncertainty-Aware Annotation for Medical Semantic Segmentation](https://mlsanity.com/paper/2308.07528/) (2023)
- [Uncertainty-aware Generalized Adaptive CycleGAN](https://mlsanity.com/paper/2102.11747/) (2021)
- [A Decoupled Uncertainty Model for MRI Segmentation Quality Estimation](https://mlsanity.com/paper/2109.02413/) (2021)
- [Lung Nodule Segmentation and Uncertain Region Prediction with an Uncertainty-Aware Attention Mechanism](https://mlsanity.com/paper/2303.08416/) (2023)
- [Uncertainty Quantification in Medical Image Segmentation with Multi-decoder U-Net](https://mlsanity.com/paper/2109.07045/) (2021)
- [Looking at the posterior: accuracy and uncertainty of neural-network predictions](https://mlsanity.com/paper/2211.14605/) (2022)
- [Attentive Continuous Generative Self-training for Unsupervised Domain Adaptive Medical Image Translation](https://mlsanity.com/paper/2305.14589/) (2023)
- [An Uncertainty-aware Hierarchical Probabilistic Network for Early Prediction, Quantification and Segmentation of Pulmonary Tumour Growth](https://mlsanity.com/paper/2104.08789/) (2021)

## Abstract (arXiv)

The accurate estimation of predictive uncertainty carries importance in medical scenarios such as lung node segmentation. Unfortunately, most existing works on predictive uncertainty do not return calibrated uncertainty estimates, which could be used in practice. In this work we exploit multi-grader annotation variability as a source of 'groundtruth' aleatoric uncertainty, which can be treated as a target in a supervised learning problem. We combine this groundtruth uncertainty with a Probabilistic U-Net and test on the LIDC-IDRI lung nodule CT dataset and MICCAI2012 prostate MRI dataset. We find that we are able to improve predictive uncertainty estimates. We also find that we can improve sample accuracy and sample diversity. In real-world applications, our method could inform doctors about the confidence of the segmentation results.