# Uncertainty Aware Training to Improve Deep Learning Model Calibration for Classification of Cardiac MR Images

arXiv: https://arxiv.org/abs/2308.15141 · HTML: https://mlsanity.com/paper/2308.15141/

## Key facts

- Cited by 7 papers in the map (data through 14 Oct 2026).
- Builds on 25 of its 40 references that are in the map.
- First posted on arXiv 29 Aug 2023.

## Builds on (most cited first, 25 of 25)

- [On Calibration of Modern Neural Networks](https://mlsanity.com/paper/1706.04599/) (2017)
- [Can You Trust Your Model's Uncertainty? Evaluating Predictive Uncertainty Under Dataset Shift](https://mlsanity.com/paper/1906.02530/) (2019)
- [Aleatoric and Epistemic Uncertainty in Machine Learning: An Introduction to Concepts and Methods](https://mlsanity.com/paper/1910.09457/) (2019)
- [Selective Classification for Deep Neural Networks](https://mlsanity.com/paper/1705.08500/) (2017)
- [A Review of Uncertainty Quantification in Deep Learning: Techniques, Applications and Challenges](https://mlsanity.com/paper/2011.06225/) (2020)
- [Evidential Deep Learning to Quantify Classification Uncertainty](https://mlsanity.com/paper/1806.01768/) (2018)
- [A Survey of Uncertainty in Deep Neural Networks](https://mlsanity.com/paper/2107.03342/) (2021)
- [Measuring Calibration in Deep Learning](https://mlsanity.com/paper/1904.01685/) (2019)
- [Uncertainty-aware Self-ensembling Model for Semi-supervised 3D Left Atrium Segmentation](https://mlsanity.com/paper/1907.07034/) (2019)
- [On Mixup Training: Improved Calibration and Predictive Uncertainty for Deep Neural Networks](https://mlsanity.com/paper/1905.11001/) (2019)
- [Aleatoric uncertainty estimation with test-time augmentation for medical image segmentation with convolutional neural networks](https://mlsanity.com/paper/1807.07356/) (2018)
- [Pitfalls of In-Domain Uncertainty Estimation and Ensembling in Deep Learning](https://mlsanity.com/paper/2002.06470/) (2020)
- [Confidence Calibration and Predictive Uncertainty Estimation for Deep Medical Image Segmentation](https://mlsanity.com/paper/1911.13273/) (2019)
- [Improving model calibration with accuracy versus uncertainty optimization](https://mlsanity.com/paper/2012.07923/) (2020)
- [Mitigating Bias in Calibration Error Estimation](https://mlsanity.com/paper/2012.08668/) (2020)
- [Calibration of Neural Networks using Splines](https://mlsanity.com/paper/2006.12800/) (2020)
- [Soft Calibration Objectives for Neural Networks](https://mlsanity.com/paper/2108.00106/) (2021)
- [Calibration of Encoder Decoder Models for Neural Machine Translation](https://mlsanity.com/paper/1903.00802/) (2019)
- [Uncertainty-Aware Deep Classifiers using Generative Models](https://mlsanity.com/paper/2006.04183/) (2020)
- [Automatic Brain Tumor Segmentation using Convolutional Neural Networks with Test-Time Augmentation](https://mlsanity.com/paper/1810.07884/) (2018)
- [Global and Local Interpretability for Cardiac MRI Classification](https://mlsanity.com/paper/1906.06188/) (2019)
- [Improving the generalizability of convolutional neural network-based segmentation on CMR images](https://mlsanity.com/paper/1907.01268/) (2019)
- [Interpretable Deep Models for Cardiac Resynchronisation Therapy Response Prediction](https://mlsanity.com/paper/2006.13811/) (2020)
- [Handling of uncertainty in medical data using machine learning and probability theory techniques: A review of 30 years (1991-2020)](https://mlsanity.com/paper/2008.10114/) (2020)
- [Uncertainty-Aware Training for Cardiac Resynchronisation Therapy Response Prediction](https://mlsanity.com/paper/2109.10641/) (2021)

## Built on by (most cited first, 7 of 7)

- [A review of uncertainty quantification in medical image analysis: probabilistic and non-probabilistic methods](https://mlsanity.com/paper/2310.06873/) (2023)
- [Average Calibration Losses for Reliable Uncertainty in Medical Image Segmentation](https://mlsanity.com/paper/2506.03942/) (2025)
- [Improving Deep Learning Model Calibration for Cardiac Applications using Deterministic Uncertainty Networks and Uncertainty-aware Training](https://mlsanity.com/paper/2405.06487/) (2024)
- [Advancing Image Classification with Discrete Diffusion Classification Modeling](https://mlsanity.com/paper/2511.20263/) (2025)
- [Multi-Rater Calibrated Segmentation Models](https://mlsanity.com/paper/2605.02437/) (2026)
- [Probing, Fusion, and Trustworthiness: A Systematic Evaluation of Foundation Model Representations for Multimodal Cancer Analysis](https://mlsanity.com/paper/2606.17115/) (2026)
- [Uncertainty-driven training for three-dimensional calibrated lung nodule classification](https://mlsanity.com/paper/2609.20905/) (2026)

## Abstract (arXiv)

Quantifying uncertainty of predictions has been identified as one way to develop more trustworthy artificial intelligence (AI) models beyond conventional reporting of performance metrics. When considering their role in a clinical decision support setting, AI classification models should ideally avoid confident wrong predictions and maximise the confidence of correct predictions. Models that do this are said to be well-calibrated with regard to confidence. However, relatively little attention has been paid to how to improve calibration when training these models, i.e., to make the training strategy uncertainty-aware. In this work we evaluate three novel uncertainty-aware training strategies comparing against two state-of-the-art approaches. We analyse performance on two different clinical applications: cardiac resynchronisation therapy (CRT) response prediction and coronary artery disease (CAD) diagnosis from cardiac magnetic resonance (CMR) images. The best-performing model in terms of both classification accuracy and the most common calibration measure, expected calibration error (ECE) was the Confidence Weight method, a novel approach that weights the loss of samples to explicitly penalise confident incorrect predictions. The method reduced the ECE by 17% for CRT response prediction and by 22% for CAD diagnosis when compared to a baseline classifier in which no uncertainty-aware strategy was included. In both applications, as well as reducing the ECE there was a slight increase in accuracy from 69% to 70% and 70% to 72% for CRT response prediction and CAD diagnosis respectively. However, our analysis showed a lack of consistency in terms of optimal models when using different calibration measures. This indicates the need for careful consideration of performance metrics when training and selecting models for complex high-risk applications in healthcare.