# Rethinking Post-Hoc Calibration in Semantic Segmentation

arXiv: https://arxiv.org/abs/2607.01902 · HTML: https://mlsanity.com/paper/2607.01902/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 29 of its 41 references that are in the map.
- First posted on arXiv 2 Jul 2026.

## Builds on (most cited first, 29 of 29)

- [PyTorch: An Imperative Style, High-Performance Deep Learning Library](https://mlsanity.com/paper/1912.01703/) (2019)
- [On Calibration of Modern Neural Networks](https://mlsanity.com/paper/1706.04599/) (2017)
- [The Cityscapes Dataset for Semantic Urban Scene Understanding](https://mlsanity.com/paper/1604.01685/) (2016)
- [Simple and Scalable Predictive Uncertainty Estimation using Deep Ensembles](https://mlsanity.com/paper/1612.01474/) (2016)
- [SegFormer: Simple and Efficient Design for Semantic Segmentation with Transformers](https://mlsanity.com/paper/2105.15203/) (2021)
- [Benchmarking Neural Network Robustness to Common Corruptions and Perturbations](https://mlsanity.com/paper/1903.12261/) (2019)
- [Optuna: A Next-generation Hyperparameter Optimization Framework](https://mlsanity.com/paper/1907.10902/) (2019)
- [Energy-based Out-of-distribution Detection](https://mlsanity.com/paper/2010.03759/) (2020)
- [Can You Trust Your Model's Uncertainty? Evaluating Predictive Uncertainty Under Dataset Shift](https://mlsanity.com/paper/1906.02530/) (2019)
- [When Does Label Smoothing Help?](https://mlsanity.com/paper/1906.02629/) (2019)
- [Generalised Dice overlap as a deep learning loss function for highly unbalanced segmentations](https://mlsanity.com/paper/1707.03237/) (2017)
- [Deep Ensembles: A Loss Landscape Perspective](https://mlsanity.com/paper/1912.02757/) (2019)
- [Beyond temperature scaling: Obtaining well-calibrated multiclass probabilities with Dirichlet calibration](https://mlsanity.com/paper/1910.12656/) (2019)
- [Calibrating Deep Neural Networks using Focal Loss](https://mlsanity.com/paper/2002.09437/) (2020)
- [Mix-n-Match: Ensemble and Compositional Methods for Uncertainty Calibration in Deep Learning](https://mlsanity.com/paper/2003.07329/) (2020)
- [Confidence Calibration and Predictive Uncertainty Estimation for Deep Medical Image Segmentation](https://mlsanity.com/paper/1911.13273/) (2019)
- [Uncertainty Quantification and Deep Ensembles](https://mlsanity.com/paper/2007.08792/) (2020)
- [Benchmarking the Robustness of Semantic Segmentation Models](https://mlsanity.com/paper/1908.05005/) (2019)
- [Local Temperature Scaling for Probability Calibration](https://mlsanity.com/paper/2008.05105/) (2020)
- [The 2024 Brain Tumor Segmentation (BraTS) Challenge: Glioma Segmentation on Post-treatment MRI](https://mlsanity.com/paper/2405.18368/) (2024)
- [Intra Order-preserving Functions for Calibration of Multi-Class Neural Networks](https://mlsanity.com/paper/2003.06820/) (2020)
- [Non-Parametric Calibration for Classification](https://mlsanity.com/paper/1906.04933/) (2019)
- [Dirichlet-based Gaussian Processes for Large-scale Calibrated Classification](https://mlsanity.com/paper/1805.10915/) (2018)
- [On Calibrating Semantic Segmentation Models: Analyses and An Algorithm](https://mlsanity.com/paper/2212.12053/) (2022)
- [ValUES: A Framework for Systematic Validation of Uncertainty Estimation in Semantic Segmentation](https://mlsanity.com/paper/2401.08501/) (2024)
- [Comparative Benchmarking of Failure Detection Methods in Medical Image Segmentation: Unveiling the Role of Confidence Aggregation](https://mlsanity.com/paper/2406.03323/) (2024)
- [Spatially-Aware Evaluation of Segmentation Uncertainty](https://mlsanity.com/paper/2506.16589/) (2025)
- [Uncertainty Calibration with Energy Based Instance-wise Scaling in the Wild Dataset](https://mlsanity.com/paper/2407.12330/) (2024)
- [We Care Each Pixel: Calibrating on Medical Segmentation Model](https://mlsanity.com/paper/2503.05107/) (2025)

## Abstract (arXiv)

Reliable confidence estimates are essential in semantic segmentation, yet modern models often remain miscalibrated. We investigate two overlooked issues in post-hoc calibration. First, adding a constant to all logits leaves softmax probabilities unchanged, but several standard calibrators depend on this arbitrary offset. In segmentation, this offset can vary across pixels or voxels, introducing spatially varying representation dependence. We characterize translation-invariant (TI) calibrators and construct TI counterparts of shift-sensitive methods. Second, calibrating with cross-entropy can degrade segmentation quality due to mismatched training and calibration objectives and limited calibration data. We investigate decision-preserving calibration under argmax- and order-preservation constraints. Since these constraints restrict affine softmax calibrators to temperature scaling, we introduce more expressive class-conditional affine calibrators that preserve decisions. Across natural-image and medical segmentation benchmarks, including corruption-based covariate shift, TI variants generally improve calibration, while decision-preserving variants prevent segmentation degradation by construction and retain strong calibration performance. Our findings provide practical design principles for post-hoc calibration in semantic segmentation.