# A Probabilistic U-Net for Segmentation of Ambiguous Images

arXiv: https://arxiv.org/abs/1806.05034 · HTML: https://mlsanity.com/paper/1806.05034/

## Key facts

- Cited by 283 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 13 Jun 2018.

## Built on by (most cited first, 30 of 283)

- [Diffusion Models for Medical Image Analysis: A Comprehensive Survey](https://mlsanity.com/paper/2211.07804/) (2022)
- [Diffusion Models for Implicit Image Segmentation Ensembles](https://mlsanity.com/paper/2112.03145/) (2021)
- [Medical Image Segmentation Review: The success of U-Net](https://mlsanity.com/paper/2211.14830/) (2022)
- [MedSegDiff-V2: Diffusion based Medical Image Segmentation with Transformer](https://mlsanity.com/paper/2301.11798/) (2023)
- [Ambiguous Medical Image Segmentation using Diffusion Models](https://mlsanity.com/paper/2304.04745/) (2023)
- [A Survey on Uncertainty Quantification Methods for Deep Learning](https://mlsanity.com/paper/2302.13425/) (2023)
- [Trustworthy clinical AI solutions: a unified review of uncertainty quantification in deep learning models for medical image analysis](https://mlsanity.com/paper/2210.03736/) (2022)
- [MUSE-VAE: Multi-Scale VAE for Environment-Aware Long Term Trajectory Prediction](https://mlsanity.com/paper/2201.07189/) (2022)
- [A Review of Uncertainty Estimation and its Application in Medical Imaging](https://mlsanity.com/paper/2302.08119/) (2023)
- [Learning Generative Vision Transformer with Energy-Based Latent Space for Saliency Prediction](https://mlsanity.com/paper/2112.13528/) (2021)
- [Uncertainty Inspired Underwater Image Enhancement](https://mlsanity.com/paper/2207.09689/) (2022)
- [A review of uncertainty quantification in medical image analysis: probabilistic and non-probabilistic methods](https://mlsanity.com/paper/2310.06873/) (2023)
- [Spatial Uncertainty-Aware Semi-Supervised Crowd Counting](https://mlsanity.com/paper/2107.13271/) (2021)
- [Stochastic Segmentation with Conditional Categorical Diffusion Models](https://mlsanity.com/paper/2303.08888/) (2023)
- [Is segmentation uncertainty useful?](https://mlsanity.com/paper/2103.16265/) (2021)
- [Transframer: Arbitrary Frame Prediction with Generative Models](https://mlsanity.com/paper/2203.09494/) (2022)
- [Uncertainty-Informed Deep Learning Models Enable High-Confidence Predictions for Digital Histopathology](https://mlsanity.com/paper/2204.04516/) (2022)
- [Inspect, Understand, Overcome: A Survey of Practical Methods for AI Safety](https://mlsanity.com/paper/2104.14235/) (2021)
- [Learning the Effect of Registration Hyperparameters with HyperMorph](https://mlsanity.com/paper/2203.16680/) (2022)
- [A comprehensive survey on deep active learning in medical image analysis](https://mlsanity.com/paper/2310.14230/) (2023)
- [Topology-Aware Uncertainty for Image Segmentation](https://mlsanity.com/paper/2306.05671/) (2023)
- [TBraTS: Trusted Brain Tumor Segmentation](https://mlsanity.com/paper/2206.09309/) (2022)
- [Embracing the Dark Knowledge: Domain Generalization Using Regularized Knowledge Distillation](https://mlsanity.com/paper/2107.02629/) (2021)
- [Variational Autoencoder for Speech Enhancement with a Noise-Aware Encoder](https://mlsanity.com/paper/2102.08706/) (2021)
- [QU-BraTS: MICCAI BraTS 2020 Challenge on Quantifying Uncertainty in Brain Tumor Segmentation - Analysis of Ranking Scores and Benchmarking Results](https://mlsanity.com/paper/2112.10074/) (2021)
- [BerDiff: Conditional Bernoulli Diffusion Model for Medical Image Segmentation](https://mlsanity.com/paper/2304.04429/) (2023)
- [A Unified Framework for U-Net Design and Analysis](https://mlsanity.com/paper/2305.19638/) (2023)
- [The Treasure Beneath Multiple Annotations: An Uncertainty-aware Edge Detector](https://mlsanity.com/paper/2303.11828/) (2023)
- [ValUES: A Framework for Systematic Validation of Uncertainty Estimation in Semantic Segmentation](https://mlsanity.com/paper/2401.08501/) (2024)
- [FlowSDF: Flow Matching for Medical Image Segmentation Using Distance Transforms](https://mlsanity.com/paper/2405.18087/) (2024)

## Abstract (arXiv)

Many real-world vision problems suffer from inherent ambiguities. In clinical applications for example, it might not be clear from a CT scan alone which particular region is cancer tissue. Therefore a group of graders typically produces a set of diverse but plausible segmentations. We consider the task of learning a distribution over segmentations given an input. To this end we propose a generative segmentation model based on a combination of a U-Net with a conditional variational autoencoder that is capable of efficiently producing an unlimited number of plausible hypotheses. We show on a lung abnormalities segmentation task and on a Cityscapes segmentation task that our model reproduces the possible segmentation variants as well as the frequencies with which they occur, doing so significantly better than published approaches. These models could have a high impact in real-world applications, such as being used as clinical decision-making algorithms accounting for multiple plausible semantic segmentation hypotheses to provide possible diagnoses and recommend further actions to resolve the present ambiguities.