# Towards objective and systematic evaluation of bias in artificial intelligence for medical imaging

arXiv: https://arxiv.org/abs/2311.02115 · HTML: https://mlsanity.com/paper/2311.02115/

## Key facts

- Cited by 4 papers in the map (data through 14 Oct 2026).
- Builds on 14 of its 35 references that are in the map.
- First posted on arXiv 3 Nov 2023.

## Builds on (most cited first, 14 of 14)

- [Learning De-biased Representations with Biased Representations](https://mlsanity.com/paper/1910.02806/) (2019)
- [Reading Race: AI Recognises Patient's Racial Identity In Medical Images](https://mlsanity.com/paper/2107.10356/) (2021)
- [Disparities in Dermatology AI: Assessments Using Diverse Clinical Images](https://mlsanity.com/paper/2111.08006/) (2021)
- [MEDFAIR: Benchmarking Fairness for Medical Imaging](https://mlsanity.com/paper/2210.01725/) (2022)
- [FairPrune: Achieving Fairness Through Pruning for Dermatological Disease Diagnosis](https://mlsanity.com/paper/2203.02110/) (2022)
- [Fairness in Cardiac MR Image Analysis: An Investigation of Bias Due to Data Imbalance in Deep Learning Based Segmentation](https://mlsanity.com/paper/2106.12387/) (2021)
- [Morpho-MNIST: Quantitative Assessment and Diagnostics for Representation Learning](https://mlsanity.com/paper/1809.10780/) (2018)
- [Improving the Fairness of Chest X-ray Classifiers](https://mlsanity.com/paper/2203.12609/) (2022)
- [Detecting Shortcut Learning for Fair Medical AI using Shortcut Testing](https://mlsanity.com/paper/2207.10384/) (2022)
- [A Study of Demographic Bias in CNN-based Brain MR Segmentation](https://mlsanity.com/paper/2208.06613/) (2022)
- [Debiasing Deep Chest X-Ray Classifiers using Intra- and Post-processing Methods](https://mlsanity.com/paper/2208.00781/) (2022)
- [Are Sex-based Physiological Differences the Cause of Gender Bias for Chest X-ray Diagnosis?](https://mlsanity.com/paper/2308.05129/) (2023)
- [The Role of Subgroup Separability in Group-Fair Medical Image Classification](https://mlsanity.com/paper/2307.02791/) (2023)
- [(Predictable) Performance Bias in Unsupervised Anomaly Detection](https://mlsanity.com/paper/2309.14198/) (2023)

## Built on by (most cited first, 4 of 4)

- [Preventing Shortcut Learning in Medical Image Analysis through Intermediate Layer Knowledge Distillation from Specialist Teachers](https://mlsanity.com/paper/2511.17421/) (2025)
- [Integration of nested cross-validation, automated hyperparameter optimization, high-performance computing to reduce and quantify the variance of test performance estimation of deep learning models](https://mlsanity.com/paper/2503.08589/) (2025)
- [Effect of Demographic Bias on Skin Lesion Classification](https://mlsanity.com/paper/2606.03214/) (2026)
- [A Neuroimaging Simulation Framework for Developing and Evaluating Causal AI](https://mlsanity.com/paper/2606.28684/) (2026)

## Abstract (arXiv)

Artificial intelligence (AI) models trained using medical images for clinical tasks often exhibit bias in the form of disparities in performance between subgroups. Since not all sources of biases in real-world medical imaging data are easily identifiable, it is challenging to comprehensively assess how those biases are encoded in models, and how capable bias mitigation methods are at ameliorating performance disparities. In this article, we introduce a novel analysis framework for systematically and objectively investigating the impact of biases in medical images on AI models. We developed and tested this framework for conducting controlled in silico trials to assess bias in medical imaging AI using a tool for generating synthetic magnetic resonance images with known disease effects and sources of bias. The feasibility is showcased by using three counterfactual bias scenarios to measure the impact of simulated bias effects on a convolutional neural network (CNN) classifier and the efficacy of three bias mitigation strategies. The analysis revealed that the simulated biases resulted in expected subgroup performance disparities when the CNN was trained on the synthetic datasets. Moreover, reweighing was identified as the most successful bias mitigation strategy for this setup, and we demonstrated how explainable AI methods can aid in investigating the manifestation of bias in the model using this framework. Developing fair AI models is a considerable challenge given that many and often unknown sources of biases can be present in medical imaging datasets. In this work, we present a novel methodology to objectively study the impact of biases and mitigation strategies on deep learning pipelines, which can support the development of clinical AI that is robust and responsible.