# Evaluating the Fairness of Deep Learning Uncertainty Estimates in Medical Image Analysis

arXiv: https://arxiv.org/abs/2303.03242 · HTML: https://mlsanity.com/paper/2303.03242/

## Key facts

- Cited by 15 papers in the map (data through 14 Oct 2026).
- Builds on 29 of its 48 references that are in the map.
- First posted on arXiv 6 Mar 2023.

## Builds on (most cited first, 29 of 29)

- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [PyTorch: An Imperative Style, High-Performance Deep Learning Library](https://mlsanity.com/paper/1912.01703/) (2019)
- [Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shift](https://mlsanity.com/paper/1502.03167/) (2015)
- [Dropout as a Bayesian Approximation: Representing Model Uncertainty in Deep Learning](https://mlsanity.com/paper/1506.02142/) (2015)
- [Simple and Scalable Predictive Uncertainty Estimation using Deep Ensembles](https://mlsanity.com/paper/1612.01474/) (2016)
- [What Uncertainties Do We Need in Bayesian Deep Learning for Computer Vision?](https://mlsanity.com/paper/1703.04977/) (2017)
- [A Survey on Bias and Fairness in Machine Learning](https://mlsanity.com/paper/1908.09635/) (2019)
- [3D U-Net: Learning Dense Volumetric Segmentation from Sparse Annotation](https://mlsanity.com/paper/1606.06650/) (2016)
- [Distributionally Robust Neural Networks for Group Shifts: On the Importance of Regularization for Worst-Case Generalization](https://mlsanity.com/paper/1911.08731/) (2019)
- [Instance Normalization: The Missing Ingredient for Fast Stylization](https://mlsanity.com/paper/1607.08022/) (2016)
- [Deep Bayesian Active Learning with Image Data](https://mlsanity.com/paper/1703.02910/) (2017)
- [A Gentle Introduction to Conformal Prediction and Distribution-Free Uncertainty Quantification](https://mlsanity.com/paper/2107.07511/) (2021)
- [Skin Lesion Analysis Toward Melanoma Detection 2018: A Challenge Hosted by the International Skin Imaging Collaboration (ISIC)](https://mlsanity.com/paper/1902.03368/) (2019)
- [Can Spatiotemporal 3D CNNs Retrace the History of 2D CNNs and ImageNet?](https://mlsanity.com/paper/1711.09577/) (2017)
- [Identifying the Best Machine Learning Algorithms for Brain Tumor Segmentation, Progression Assessment, and Overall Survival Prediction in the BRATS Challenge](https://mlsanity.com/paper/1811.02629/) (2018)
- [Simple data balancing achieves competitive worst-group-accuracy](https://mlsanity.com/paper/2110.14503/) (2021)
- [Understanding Measures of Uncertainty for Adversarial Example Detection](https://mlsanity.com/paper/1803.08533/) (2018)
- [Exploring Uncertainty Measures in Deep Networks for Multiple Sclerosis Lesion Detection and Segmentation](https://mlsanity.com/paper/1808.01200/) (2018)
- [Fairness in Deep Learning: A Computational Perspective](https://mlsanity.com/paper/1908.08843/) (2019)
- [Assessing Reliability and Challenges of Uncertainty Estimations for Medical Image Segmentation](https://mlsanity.com/paper/1907.03338/) (2019)
- [MEDFAIR: Benchmarking Fairness for Medical Imaging](https://mlsanity.com/paper/2210.01725/) (2022)
- [Fairness in Cardiac MR Image Analysis: An Investigation of Bias Due to Data Imbalance in Deep Learning Based Segmentation](https://mlsanity.com/paper/2106.12387/) (2021)
- [Leveling Down in Computer Vision: Pareto Inefficiencies in Fair Deep Classifiers](https://mlsanity.com/paper/2203.04913/) (2022)
- [QU-BraTS: MICCAI BraTS 2020 Challenge on Quantifying Uncertainty in Brain Tumor Segmentation - Analysis of Ranking Scores and Benchmarking Results](https://mlsanity.com/paper/2112.10074/) (2021)
- [Quantifying and Leveraging Classification Uncertainty for Chest Radiograph Assessment](https://mlsanity.com/paper/1906.07775/) (2019)
- [A Study of Demographic Bias in CNN-based Brain MR Segmentation](https://mlsanity.com/paper/2208.06613/) (2022)
- [Evaluating Fairness Metrics in the Presence of Dataset Bias](https://mlsanity.com/paper/1809.09245/) (2018)
- [RS-Net: Regression-Segmentation 3D CNN for Synthesis of Full Resolution Missing Brain MRI in the Presence of Tumours](https://mlsanity.com/paper/1807.10972/) (2018)

## Built on by (most cited first, 15 of 15)

- [A review of uncertainty quantification in medical image analysis: probabilistic and non-probabilistic methods](https://mlsanity.com/paper/2310.06873/) (2023)
- [Uncertainty-based Fairness Measures](https://mlsanity.com/paper/2312.11299/) (2023)
- [A Short Survey on Importance Weighting for Machine Learning](https://mlsanity.com/paper/2403.10175/) (2024)
- [Conformal Prediction Sets Can Cause Disparate Impact](https://mlsanity.com/paper/2410.01888/) (2024)
- [Debiasing Counterfactuals In the Presence of Spurious Correlations](https://mlsanity.com/paper/2308.10984/) (2023)
- [U-Fair: Uncertainty-based Multimodal Multitask Learning for Fairer Depression Detection](https://mlsanity.com/paper/2501.09687/) (2025)
- [Understanding Fairness Surrogate Functions in Algorithmic Fairness](https://mlsanity.com/paper/2310.11211/) (2023)
- [Are demographically invariant models and representations in medical imaging fair?](https://mlsanity.com/paper/2305.01397/) (2023)
- [Bridging Formal Language with Chain-of-Thought Reasoning to Geometry Problem Solving](https://mlsanity.com/paper/2508.09099/) (2025)
- [Is Your Model Fairly Certain? Uncertainty-Aware Fairness Evaluation for LLMs](https://mlsanity.com/paper/2505.23996/) (2025)
- [Evaluating the Fairness of Neural Collapse in Medical Image Classification](https://mlsanity.com/paper/2407.05843/) (2024)
- [OxEnsemble: Fair Ensembles for Low-Data Classification](https://mlsanity.com/paper/2512.09665/) (2025)
- [EQPO: Equitable Group Relative Policy Optimization for Clinical Reasoning](https://mlsanity.com/paper/2510.19893/) (2025)
- [Beyond Predictive Fairness: Quantifying Attribution Consistency Across Demographic Groups in Diabetic Retinopathy Screening](https://mlsanity.com/paper/2608.18759/) (2026)
- [Learning the Robustness Mechanism with Bilevel Optimization](https://mlsanity.com/paper/2609.35541/) (2026)

## Abstract (arXiv)

Although deep learning (DL) models have shown great success in many medical image analysis tasks, deployment of the resulting models into real clinical contexts requires: (1) that they exhibit robustness and fairness across different sub-populations, and (2) that the confidence in DL model predictions be accurately expressed in the form of uncertainties. Unfortunately, recent studies have indeed shown significant biases in DL models across demographic subgroups (e.g., race, sex, age) in the context of medical image analysis, indicating a lack of fairness in the models. Although several methods have been proposed in the ML literature to mitigate a lack of fairness in DL models, they focus entirely on the absolute performance between groups without considering their effect on uncertainty estimation. In this work, we present the first exploration of the effect of popular fairness models on overcoming biases across subgroups in medical image analysis in terms of bottom-line performance, and their effects on uncertainty quantification. We perform extensive experiments on three different clinically relevant tasks: (i) skin lesion classification, (ii) brain tumour segmentation, and (iii) Alzheimer's disease clinical score regression. Our results indicate that popular ML methods, such as data-balancing and distributionally robust optimization, succeed in mitigating fairness issues in terms of the model performances for some of the tasks. However, this can come at the cost of poor uncertainty estimates associated with the model predictions. This tradeoff must be mitigated if fairness models are to be adopted in medical image analysis.