# Conformal Performance Range Prediction for Segmentation Output Quality Control

arXiv: https://arxiv.org/abs/2407.13307 · HTML: https://mlsanity.com/paper/2407.13307/

## Key facts

- Cited by 8 papers in the map (data through 14 Oct 2026).
- Builds on 16 of its 34 references that are in the map.
- First posted on arXiv 18 Jul 2024.

## Builds on (most cited first, 16 of 16)

- [On Calibration of Modern Neural Networks](https://mlsanity.com/paper/1706.04599/) (2017)
- [Dropout as a Bayesian Approximation: Representing Model Uncertainty in Deep Learning](https://mlsanity.com/paper/1506.02142/) (2015)
- [Simple and Scalable Predictive Uncertainty Estimation using Deep Ensembles](https://mlsanity.com/paper/1612.01474/) (2016)
- [A Gentle Introduction to Conformal Prediction and Distribution-Free Uncertainty Quantification](https://mlsanity.com/paper/2107.07511/) (2021)
- [Conformal prediction beyond exchangeability](https://mlsanity.com/paper/2202.13415/) (2022)
- [A Probabilistic U-Net for Segmentation of Ambiguous Images](https://mlsanity.com/paper/1806.05034/) (2018)
- [Medical Image Segmentation Review: The success of U-Net](https://mlsanity.com/paper/2211.14830/) (2022)
- [PHiSeg: Capturing Uncertainty in Medical Image Segmentation](https://mlsanity.com/paper/1906.04045/) (2019)
- [Image-to-Image Regression with Distribution-Free Uncertainty Quantification and Applications in Imaging](https://mlsanity.com/paper/2202.05265/) (2022)
- [Residual Pattern Learning for Pixel-wise Out-of-Distribution Detection in Semantic Segmentation](https://mlsanity.com/paper/2211.14512/) (2022)
- [Reverse Classification Accuracy: Predicting Segmentation Performance in the Absence of Ground Truth](https://mlsanity.com/paper/1702.03407/) (2017)
- [ValUES: A Framework for Systematic Validation of Uncertainty Estimation in Semantic Segmentation](https://mlsanity.com/paper/2401.08501/) (2024)
- [Real-time Prediction of Segmentation Quality](https://mlsanity.com/paper/1806.06244/) (2018)
- [Estimating Model Performance under Domain Shifts with Class-Specific Confidence Scores](https://mlsanity.com/paper/2207.09957/) (2022)
- [Automated quantification of myocardial tissue characteristics from native T1 mapping using neural networks with Bayesian inference for uncertainty-based quality-control](https://mlsanity.com/paper/2001.11711/) (2020)
- [Automatic quality control framework for more reliable integration of machine learning-based image segmentation into medical workflows](https://mlsanity.com/paper/2112.03277/) (2021)

## Built on by (most cited first, 8 of 8)

- [Conformal Risk Control for Semantic Uncertainty Quantification in Computed Tomography](https://mlsanity.com/paper/2503.00136/) (2025)
- [A Review of Bayesian Uncertainty Quantification in Deep Probabilistic Image Segmentation](https://mlsanity.com/paper/2411.16370/) (2024)
- [ConfIC-RCA: Statistically Grounded Efficient Estimation of Segmentation Quality](https://mlsanity.com/paper/2503.04522/) (2025)
- [Towards Ground-truth-free Evaluation of Any Segmentation in Medical Images](https://mlsanity.com/paper/2409.14874/) (2024)
- [CONSIGN: Conformal Segmentation Informed by Spatial Groupings via Decomposition](https://mlsanity.com/paper/2505.14113/) (2025)
- [CUTE-MRI: Conformalized Uncertainty-based framework for Time-adaptivE MRI](https://mlsanity.com/paper/2508.14952/) (2025)
- [A Critical Perspective on Finite Sample Conformal Prediction Theory in Medical Applications](https://mlsanity.com/paper/2512.14727/) (2025)
- [Bound-Aware Per-Organ Recall Risk Control for Multi-Organ CT Segmentation under Clinical Domain Shift](https://mlsanity.com/paper/2608.18193/) (2026)

## Abstract (arXiv)

Recent works have introduced methods to estimate segmentation performance without ground truth, relying solely on neural network softmax outputs. These techniques hold potential for intuitive output quality control. However, such performance estimates rely on calibrated softmax outputs, which is often not the case in modern neural networks. Moreover, the estimates do not take into account inherent uncertainty in segmentation tasks. These limitations may render precise performance predictions unattainable, restricting the practical applicability of performance estimation methods. To address these challenges, we develop a novel approach for predicting performance ranges with statistical guarantees of containing the ground truth with a user specified probability. Our method leverages sampling-based segmentation uncertainty estimation to derive heuristic performance ranges, and applies split conformal prediction to transform these estimates into rigorous prediction ranges that meet the desired guarantees. We demonstrate our approach on the FIVES retinal vessel segmentation dataset and compare five commonly used sampling-based uncertainty estimation techniques. Our results show that it is possible to achieve the desired coverage with small prediction ranges, highlighting the potential of performance range prediction as a valuable tool for output quality control.