# Stochastic Segmentation Networks: Modelling Spatially Correlated Aleatoric Uncertainty

arXiv: https://arxiv.org/abs/2006.06015 · HTML: https://mlsanity.com/paper/2006.06015/

## Key facts

- Cited by 101 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 10 Jun 2020.

## Built on by (most cited first, 30 of 101)

- [Diffusion Models for Implicit Image Segmentation Ensembles](https://mlsanity.com/paper/2112.03145/) (2021)
- [Multi-modal Learning with Missing Modality via Shared-Specific Feature Modelling](https://mlsanity.com/paper/2307.14126/) (2023)
- [Ambiguous Medical Image Segmentation using Diffusion Models](https://mlsanity.com/paper/2304.04745/) (2023)
- [Bayes' Rays: Uncertainty Quantification for Neural Radiance Fields](https://mlsanity.com/paper/2309.03185/) (2023)
- [Trustworthy clinical AI solutions: a unified review of uncertainty quantification in deep learning models for medical image analysis](https://mlsanity.com/paper/2210.03736/) (2022)
- [Stochastic Concept Bottleneck Models](https://mlsanity.com/paper/2406.19272/) (2024)
- [A review of uncertainty quantification in medical image analysis: probabilistic and non-probabilistic methods](https://mlsanity.com/paper/2310.06873/) (2023)
- [Stochastic Segmentation with Conditional Categorical Diffusion Models](https://mlsanity.com/paper/2303.08888/) (2023)
- [Is segmentation uncertainty useful?](https://mlsanity.com/paper/2103.16265/) (2021)
- [Correlated Input-Dependent Label Noise in Large-Scale Image Classification](https://mlsanity.com/paper/2105.10305/) (2021)
- [Confidence-based Out-of-Distribution Detection: A Comparative Study and Analysis](https://mlsanity.com/paper/2107.02568/) (2021)
- [Spatially Varying Label Smoothing: Capturing Uncertainty from Expert Annotations](https://mlsanity.com/paper/2104.05788/) (2021)
- [QU-BraTS: MICCAI BraTS 2020 Challenge on Quantifying Uncertainty in Brain Tumor Segmentation - Analysis of Ranking Scores and Benchmarking Results](https://mlsanity.com/paper/2112.10074/) (2021)
- [On Calibrating Semantic Segmentation Models: Analyses and An Algorithm](https://mlsanity.com/paper/2212.12053/) (2022)
- [ValUES: A Framework for Systematic Validation of Uncertainty Estimation in Semantic Segmentation](https://mlsanity.com/paper/2401.08501/) (2024)
- [Tyche: Stochastic In-Context Learning for Medical Image Segmentation](https://mlsanity.com/paper/2401.13650/) (2024)
- [Detecting when pre-trained nnU-Net models fail silently for Covid-19 lung lesion segmentation](https://mlsanity.com/paper/2107.05975/) (2021)
- [Using Soft Labels to Model Uncertainty in Medical Image Segmentation](https://mlsanity.com/paper/2109.12622/) (2021)
- [Collaborative Uncertainty in Multi-Agent Trajectory Forecasting](https://mlsanity.com/paper/2110.13947/) (2021)
- [Uncertainty Quantification via Neural Posterior Principal Components](https://mlsanity.com/paper/2309.15533/) (2023)
- [Heteroscedastic Uncertainty Estimation Framework for Unsupervised Registration](https://mlsanity.com/paper/2312.00836/) (2023)
- [Collaborative Uncertainty Benefits Multi-Agent Multi-Modal Trajectory Forecasting](https://mlsanity.com/paper/2207.05195/) (2022)
- [Uncertainty-aware Multi-modal Learning via Cross-modal Random Network Prediction](https://mlsanity.com/paper/2207.10851/) (2022)
- [Modeling Multimodal Aleatoric Uncertainty in Segmentation with Mixture of Stochastic Experts](https://mlsanity.com/paper/2212.07328/) (2022)
- [That Label's Got Style: Handling Label Style Bias for Uncertain Image Segmentation](https://mlsanity.com/paper/2303.15850/) (2023)
- [Anatomically-aware Uncertainty for Semi-supervised Image Segmentation](https://mlsanity.com/paper/2310.16099/) (2023)
- [QUBIQ: Uncertainty Quantification for Biomedical Image Segmentation Challenge](https://mlsanity.com/paper/2405.18435/) (2024)
- [Modeling Annotator Preference and Stochastic Annotation Error for Medical Image Segmentation](https://mlsanity.com/paper/2111.13410/) (2021)
- [Comparative Benchmarking of Failure Detection Methods in Medical Image Segmentation: Unveiling the Role of Confidence Aggregation](https://mlsanity.com/paper/2406.03323/) (2024)
- [Exploring Feature Representation Learning for Semi-supervised Medical Image Segmentation](https://mlsanity.com/paper/2111.10989/) (2021)

## Abstract (arXiv)

In image segmentation, there is often more than one plausible solution for a given input. In medical imaging, for example, experts will often disagree about the exact location of object boundaries. Estimating this inherent uncertainty and predicting multiple plausible hypotheses is of great interest in many applications, yet this ability is lacking in most current deep learning methods. In this paper, we introduce stochastic segmentation networks (SSNs), an efficient probabilistic method for modelling aleatoric uncertainty with any image segmentation network architecture. In contrast to approaches that produce pixel-wise estimates, SSNs model joint distributions over entire label maps and thus can generate multiple spatially coherent hypotheses for a single image. By using a low-rank multivariate normal distribution over the logit space to model the probability of the label map given the image, we obtain a spatially consistent probability distribution that can be efficiently computed by a neural network without any changes to the underlying architecture. We tested our method on the segmentation of real-world medical data, including lung nodules in 2D CT and brain tumours in 3D multimodal MRI scans. SSNs outperform state-of-the-art for modelling correlated uncertainty in ambiguous images while being much simpler, more flexible, and more efficient.