# Conformal Semantic Image Segmentation: Post-hoc Quantification of Predictive Uncertainty

arXiv: https://arxiv.org/abs/2405.05145 · HTML: https://mlsanity.com/paper/2405.05145/

## Key facts

- Cited by 20 papers in the map (data through 14 Oct 2026).
- Builds on 45 of its 66 references that are in the map.
- First posted on arXiv 16 Apr 2024.

## Builds on (most cited first, 30 of 45)

- [PyTorch: An Imperative Style, High-Performance Deep Learning Library](https://mlsanity.com/paper/1912.01703/) (2019)
- [U-Net: Convolutional Networks for Biomedical Image Segmentation](https://mlsanity.com/paper/1505.04597/) (2015)
- [End-to-End Object Detection with Transformers](https://mlsanity.com/paper/2005.12872/) (2020)
- [On Calibration of Modern Neural Networks](https://mlsanity.com/paper/1706.04599/) (2017)
- [The Cityscapes Dataset for Semantic Urban Scene Understanding](https://mlsanity.com/paper/1604.01685/) (2016)
- [SegFormer: Simple and Efficient Design for Semantic Segmentation with Transformers](https://mlsanity.com/paper/2105.15203/) (2021)
- [Pyramid Scene Parsing Network](https://mlsanity.com/paper/1612.01105/) (2016)
- [What Uncertainties Do We Need in Bayesian Deep Learning for Computer Vision?](https://mlsanity.com/paper/1703.04977/) (2017)
- [Semantic Understanding of Scenes through the ADE20K Dataset](https://mlsanity.com/paper/1608.05442/) (2016)
- [Aleatoric and Epistemic Uncertainty in Machine Learning: An Introduction to Concepts and Methods](https://mlsanity.com/paper/1910.09457/) (2019)
- [A Gentle Introduction to Conformal Prediction and Distribution-Free Uncertainty Quantification](https://mlsanity.com/paper/2107.07511/) (2021)
- [ResNeSt: Split-Attention Networks](https://mlsanity.com/paper/2004.08955/) (2020)
- [Classification with Valid and Adaptive Coverage](https://mlsanity.com/paper/2006.02544/) (2020)
- [Conformal Risk Control](https://mlsanity.com/paper/2208.02814/) (2022)
- [Distribution-Free, Risk-Controlling Prediction Sets](https://mlsanity.com/paper/2101.02703/) (2021)
- [Least Ambiguous Set-Valued Classifiers with Bounded Error Levels](https://mlsanity.com/paper/1609.00451/) (2016)
- [Learn then Test: Calibrating Predictive Algorithms to Achieve Risk Control](https://mlsanity.com/paper/2110.01052/) (2021)
- [LoveDA: A Remote Sensing Land-Cover Dataset for Domain Adaptive Semantic Segmentation](https://mlsanity.com/paper/2110.08733/) (2021)
- [Bayesian SegNet: Model Uncertainty in Deep Convolutional Encoder-Decoder Architectures for Scene Understanding](https://mlsanity.com/paper/1511.02680/) (2015)
- [Deep Deterministic Uncertainty: A Simple Baseline](https://mlsanity.com/paper/2102.11582/) (2021)
- [Addressing Failure Prediction by Learning Model Confidence](https://mlsanity.com/paper/1910.04851/) (2019)
- [SegmentMeIfYouCan: A Benchmark for Anomaly Segmentation](https://mlsanity.com/paper/2104.14812/) (2021)
- [Entropy Maximization and Meta Classification for Out-Of-Distribution Detection in Semantic Segmentation](https://mlsanity.com/paper/2012.06575/) (2020)
- [Image-to-Image Regression with Distribution-Free Uncertainty Quantification and Applications in Imaging](https://mlsanity.com/paper/2202.05265/) (2022)
- [Nested conformal prediction and quantile out-of-bag ensemble methods](https://mlsanity.com/paper/1910.10562/) (2019)
- [Distribution-free binary classification: prediction sets, confidence intervals and calibration](https://mlsanity.com/paper/2006.10564/) (2020)
- [PAC Confidence Sets for Deep Neural Networks via Calibrated Prediction](https://mlsanity.com/paper/2001.00106/) (2019)
- [How to Trust Your Diffusion Model: A Convex Optimization Approach to Conformal Risk Control](https://mlsanity.com/paper/2302.03791/) (2023)
- [Top-label calibration and multiclass-to-binary reductions](https://mlsanity.com/paper/2107.08353/) (2021)
- [Is segmentation uncertainty useful?](https://mlsanity.com/paper/2103.16265/) (2021)

## Built on by (most cited first, 20 of 20)

- [Conformal Prediction: A Data Perspective](https://mlsanity.com/paper/2410.06494/) (2024)
- [Conformal Prediction for Image Segmentation Using Morphological Prediction Sets](https://mlsanity.com/paper/2503.05618/) (2025)
- [Conformal Risk Control for Semantic Uncertainty Quantification in Computed Tomography](https://mlsanity.com/paper/2503.00136/) (2025)
- [Conformal confidence sets for biomedical image segmentation](https://mlsanity.com/paper/2410.03406/) (2024)
- [A Review of Bayesian Uncertainty Quantification in Deep Probabilistic Image Segmentation](https://mlsanity.com/paper/2411.16370/) (2024)
- [WQLCP: Weighted Adaptive Conformal Prediction for Robust Uncertainty Quantification Under Distribution Shifts](https://mlsanity.com/paper/2505.19587/) (2025)
- [Efficient Robust Conformal Prediction via Lipschitz-Bounded Networks](https://mlsanity.com/paper/2506.05434/) (2025)
- [COMPASS: Robust Feature Conformal Prediction for Medical Segmentation Metrics](https://mlsanity.com/paper/2509.22240/) (2025)
- [CONSIGN: Conformal Segmentation Informed by Spatial Groupings via Decomposition](https://mlsanity.com/paper/2505.14113/) (2025)
- [Adaptive Conformal Guidance for Learning under Uncertainty](https://mlsanity.com/paper/2502.16736/) (2025)
- [Conformal Segmentation in Industrial Surface Defect Detection with Statistical Guarantees](https://mlsanity.com/paper/2504.17721/) (2025)
- [Image Super-Resolution with Guarantees via Conformalized Generative Models](https://mlsanity.com/paper/2502.09664/) (2025)
- [Controlling False Positives in Image Segmentation via Conformal Prediction](https://mlsanity.com/paper/2511.15406/) (2025)
- [Conformal Prediction Sets for Instance Segmentation](https://mlsanity.com/paper/2602.10045/) (2026)
- [Aerial Image Classification in Scarce and Unconstrained Environments via Conformal Prediction](https://mlsanity.com/paper/2504.17655/) (2025)
- [Learning to Navigate Under Imperfect Perception: Conformalised Segmentation for Safe Reinforcement Learning](https://mlsanity.com/paper/2510.18485/) (2025)
- [Class Adaptive Conformal Training](https://mlsanity.com/paper/2601.09522/) (2026)
- [Anatomically-aware conformal prediction for medical image segmentation with random walks](https://mlsanity.com/paper/2601.18997/) (2026)
- [Occupancy-based Quantile Risk Control](https://mlsanity.com/paper/2609.03104/) (2026)
- [Trust Before Autonomy: Rethinking Agentic Remote Sensing for Crisis Response](https://mlsanity.com/paper/2610.18789/) (2026)

## Abstract (arXiv)

We propose a post-hoc, computationally lightweight method to quantify predictive uncertainty in semantic image segmentation. Our approach uses conformal prediction to generate statistically valid prediction sets that are guaranteed to include the ground-truth segmentation mask at a predefined confidence level. We introduce a novel visualization technique of conformalized predictions based on heatmaps, and provide metrics to assess their empirical validity. We demonstrate the effectiveness of our approach on well-known benchmark datasets and image segmentation prediction models, and conclude with practical insights.