# Average Calibration Error: A Differentiable Loss for Improved Reliability in Image Segmentation

arXiv: https://arxiv.org/abs/2403.06759 · HTML: https://mlsanity.com/paper/2403.06759/

## Key facts

- Cited by 6 papers in the map (data through 14 Oct 2026).
- Builds on 15 of its 24 references that are in the map.
- First posted on arXiv 11 Mar 2024.

## Builds on (most cited first, 15 of 15)

- [U-Net: Convolutional Networks for Biomedical Image Segmentation](https://mlsanity.com/paper/1505.04597/) (2015)
- [On Calibration of Modern Neural Networks](https://mlsanity.com/paper/1706.04599/) (2017)
- [The RSNA-ASNR-MICCAI BraTS 2021 Benchmark on Brain Tumor Segmentation and Radiogenomic Classification](https://mlsanity.com/paper/2107.02314/) (2021)
- [MONAI: An open-source framework for deep learning in healthcare](https://mlsanity.com/paper/2211.02701/) (2022)
- [Beyond temperature scaling: Obtaining well-calibrated multiclass probabilities with Dirichlet calibration](https://mlsanity.com/paper/1910.12656/) (2019)
- [Confidence Calibration and Predictive Uncertainty Estimation for Deep Medical Image Segmentation](https://mlsanity.com/paper/1911.13273/) (2019)
- [Local Temperature Scaling for Probability Calibration](https://mlsanity.com/paper/2008.05105/) (2020)
- [A Stitch in Time Saves Nine: A Train-Time Regularizing Loss for Improved Neural Network Calibration](https://mlsanity.com/paper/2203.13834/) (2022)
- [Improved Trainable Calibration Method for Neural Networks on Medical Imaging Classification](https://mlsanity.com/paper/2009.04057/) (2020)
- [Meta-Calibration: Learning of Model Calibration Using Differentiable Expected Calibration Error](https://mlsanity.com/paper/2106.09613/) (2021)
- [Class-Distribution-Aware Calibration for Long-Tailed Visual Recognition](https://mlsanity.com/paper/2109.05263/) (2021)
- [Calibrating the Dice loss to handle neural network overconfidence for biomedical image segmentation](https://mlsanity.com/paper/2111.00528/) (2021)
- [Trust your neighbours: Penalty-based constraints for model calibration](https://mlsanity.com/paper/2303.06268/) (2023)
- [Neighbor-Aware Calibration of Segmentation Networks with Penalty-Based Constraints](https://mlsanity.com/paper/2401.14487/) (2024)
- [Generalized Wasserstein Dice Loss, Test-time Augmentation, and Transformers for the BraTS 2021 challenge](https://mlsanity.com/paper/2112.13054/) (2021)

## Built on by (most cited first, 6 of 6)

- [Calibration and Uncertainty for multiRater Volume Assessment in multiorgan Segmentation (CURVAS) challenge results](https://mlsanity.com/paper/2505.08685/) (2025)
- [Average Calibration Losses for Reliable Uncertainty in Medical Image Segmentation](https://mlsanity.com/paper/2506.03942/) (2025)
- [Flow Stochastic Segmentation Networks](https://mlsanity.com/paper/2507.18838/) (2025)
- [OOD-SEG: Exploiting out-of-distribution detection techniques for learning image segmentation from sparse multi-class positive-only annotations](https://mlsanity.com/paper/2411.09553/) (2024)
- [Rethinking Glaucoma Calibration: Voting-Based Binocular and Metadata Integration](https://mlsanity.com/paper/2503.18642/) (2025)
- [Beyond scalar losses: calibrating segmentation models via gradient vector field surgery](https://mlsanity.com/paper/2607.14338/) (2026)

## Abstract (arXiv)

Deep neural networks for medical image segmentation often produce overconfident results misaligned with empirical observations. Such miscalibration, challenges their clinical translation. We propose to use marginal L1 average calibration error (mL1-ACE) as a novel auxiliary loss function to improve pixel-wise calibration without compromising segmentation quality. We show that this loss, despite using hard binning, is directly differentiable, bypassing the need for approximate but differentiable surrogate or soft binning approaches. Our work also introduces the concept of dataset reliability histograms which generalises standard reliability diagrams for refined visual assessment of calibration in semantic segmentation aggregated at the dataset level. Using mL1-ACE, we reduce average and maximum calibration error by 45% and 55% respectively, maintaining a Dice score of 87% on the BraTS 2021 dataset. We share our code here: https://github.com/cai4cai/ACE-DLIRIS