# CheXclusion: Fairness gaps in deep chest X-ray classifiers

arXiv: https://arxiv.org/abs/2003.00827 · HTML: https://mlsanity.com/paper/2003.00827/

## Key facts

- Cited by 153 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 14 Feb 2020.

## Built on by (most cited first, 30 of 153)

- [RadGraph: Extracting Clinical Entities and Relations from Radiology Reports](https://mlsanity.com/paper/2106.14463/) (2021)
- [Reading Race: AI Recognises Patient's Racial Identity In Medical Images](https://mlsanity.com/paper/2107.10356/) (2021)
- [A Unified Survey on Anomaly, Novelty, Open-Set, and Out-of-Distribution Detection: Solutions and Future Challenges](https://mlsanity.com/paper/2110.14051/) (2021)
- [AUC Maximization in the Era of Big Data and AI: A Survey](https://mlsanity.com/paper/2203.15046/) (2022)
- [Fair Conformal Predictors for Applications in Medical Imaging](https://mlsanity.com/paper/2109.04392/) (2021)
- [FairCLIP: Harnessing Fairness in Vision-Language Learning](https://mlsanity.com/paper/2403.19949/) (2024)
- [MEDFAIR: Benchmarking Fairness for Medical Imaging](https://mlsanity.com/paper/2210.01725/) (2022)
- [Delving into Masked Autoencoders for Multi-Label Thorax Disease Classification](https://mlsanity.com/paper/2210.12843/) (2022)
- [Meaningfully Debugging Model Mistakes using Conceptual Counterfactual Explanations](https://mlsanity.com/paper/2106.12723/) (2021)
- [High Fidelity Image Counterfactuals with Probabilistic Causal Models](https://mlsanity.com/paper/2306.15764/) (2023)
- [Improving the Fairness of Chest X-ray Classifiers](https://mlsanity.com/paper/2203.12609/) (2022)
- [M-FLAG: Medical Vision-Language Pre-training with Frozen Language Models and Latent Space Geometry Optimization](https://mlsanity.com/paper/2307.08347/) (2023)
- [An Empirical Framework for Domain Generalization in Clinical Settings](https://mlsanity.com/paper/2103.11163/) (2021)
- [RadFusion: Benchmarking Performance and Fairness for Multimodal Pulmonary Embolism Detection from CT and EHR](https://mlsanity.com/paper/2111.11665/) (2021)
- [Addressing Fairness Issues in Deep Learning-Based Medical Image Analysis: A Systematic Review](https://mlsanity.com/paper/2209.13177/) (2022)
- [Risk of Bias in Chest Radiography Deep Learning Foundation Models](https://mlsanity.com/paper/2209.02965/) (2022)
- [Estimating and Improving Fairness with Adversarial Learning](https://mlsanity.com/paper/2103.04243/) (2021)
- [EVA-X: A Foundation Model for General Chest X-ray Analysis with Self-supervised Learning](https://mlsanity.com/paper/2405.05237/) (2024)
- [Demographic Bias of Expert-Level Vision-Language Foundation Models in Medical Imaging](https://mlsanity.com/paper/2402.14815/) (2024)
- [Towards long-tailed, multi-label disease classification from chest X-ray: Overview of the CXR-LT challenge](https://mlsanity.com/paper/2310.16112/) (2023)
- [Algorithm Fairness in AI for Medicine and Healthcare](https://mlsanity.com/paper/2110.00603/) (2021)
- [Technical Challenges for Training Fair Neural Networks](https://mlsanity.com/paper/2102.06764/) (2021)
- [Label-Assemble: Leveraging Multiple Datasets with Partial Labels](https://mlsanity.com/paper/2109.12265/) (2021)
- [Detecting Shortcuts in Medical Images -- A Case Study in Chest X-rays](https://mlsanity.com/paper/2211.04279/) (2022)
- [Intermediate Layers Matter in Momentum Contrastive Self Supervised Learning](https://mlsanity.com/paper/2110.14805/) (2021)
- [Training Subset Selection for Weak Supervision](https://mlsanity.com/paper/2206.02914/) (2022)
- [FairAdaBN: Mitigating unfairness with adaptive batch normalization and its application to dermatological disease classification](https://mlsanity.com/paper/2303.08325/) (2023)
- [CIRCLe: Color Invariant Representation Learning for Unbiased Classification of Skin Lesions](https://mlsanity.com/paper/2208.13528/) (2022)
- [Net benefit, calibration, threshold selection, and training objectives for algorithmic fairness in healthcare](https://mlsanity.com/paper/2202.01906/) (2022)
- [Unlocking Accuracy and Fairness in Differentially Private Image Classification](https://mlsanity.com/paper/2308.10888/) (2023)

## Abstract (arXiv)

Machine learning systems have received much attention recently for their ability to achieve expert-level performance on clinical tasks, particularly in medical imaging. Here, we examine the extent to which state-of-the-art deep learning classifiers trained to yield diagnostic labels from X-ray images are biased with respect to protected attributes. We train convolution neural networks to predict 14 diagnostic labels in 3 prominent public chest X-ray datasets: MIMIC-CXR, Chest-Xray8, CheXpert, as well as a multi-site aggregation of all those datasets. We evaluate the TPR disparity -- the difference in true positive rates (TPR) -- among different protected attributes such as patient sex, age, race, and insurance type as a proxy for socioeconomic status. We demonstrate that TPR disparities exist in the state-of-the-art classifiers in all datasets, for all clinical tasks, and all subgroups. A multi-source dataset corresponds to the smallest disparities, suggesting one way to reduce bias. We find that TPR disparities are not significantly correlated with a subgroup's proportional disease burden. As clinical models move from papers to products, we encourage clinical decision makers to carefully audit for algorithmic disparities prior to deployment. Our code can be found at, https://github.com/LalehSeyyed/CheXclusion