# Potential sources of dataset bias complicate investigation of underdiagnosis by machine learning algorithms

arXiv: https://arxiv.org/abs/2201.07856 · HTML: https://mlsanity.com/paper/2201.07856/

## Key facts

- Cited by 19 papers in the map (data through 14 Oct 2026).
- Builds on 2 of its 12 references that are in the map.
- First posted on arXiv 19 Jan 2022.

## Builds on (most cited first, 2 of 2)

- [CheXpert: A Large Chest Radiograph Dataset with Uncertainty Labels and Expert Comparison](https://mlsanity.com/paper/1901.07031/) (2019)
- [Causality matters in medical imaging](https://mlsanity.com/paper/1912.08142/) (2019)

## Built on by (most cited first, 19 of 19)

- [High Fidelity Image Counterfactuals with Probabilistic Causal Models](https://mlsanity.com/paper/2306.15764/) (2023)
- [Risk of Bias in Chest Radiography Deep Learning Foundation Models](https://mlsanity.com/paper/2209.02965/) (2022)
- [Advancing human-centric AI for robust X-ray analysis through holistic self-supervised learning](https://mlsanity.com/paper/2405.01469/) (2024)
- [Are Sex-based Physiological Differences the Cause of Gender Bias for Chest X-ray Diagnosis?](https://mlsanity.com/paper/2308.05129/) (2023)
- [The Role of Subgroup Separability in Group-Fair Medical Image Classification](https://mlsanity.com/paper/2307.02791/) (2023)
- [No Fair Lunch: A Causal Perspective on Dataset Bias in Machine Learning for Medical Imaging](https://mlsanity.com/paper/2307.16526/) (2023)
- [Faces of Fairness: Examining Bias in Facial Expression Recognition Datasets and Models](https://mlsanity.com/paper/2502.11049/) (2025)
- [Robustness Stress Testing in Medical Image Classification](https://mlsanity.com/paper/2308.06889/) (2023)
- [An investigation of challenges encountered when specifying training data and runtime monitors for safety critical ML applications](https://mlsanity.com/paper/2301.13476/) (2023)
- [Trustworthy and Responsible AI for Human-Centric Autonomous Decision-Making Systems](https://mlsanity.com/paper/2408.15550/) (2024)
- [Gender Fairness of Machine Learning Algorithms for Pain Detection](https://mlsanity.com/paper/2506.11132/) (2025)
- [Out-of-Distribution Detection and Data Drift Monitoring using Statistical Process Control](https://mlsanity.com/paper/2402.08088/) (2024)
- [A Dual-Perspective Approach to Evaluating Feature Attribution Methods](https://mlsanity.com/paper/2308.08949/) (2023)
- [Addressing Weak Decision Boundaries in Image Classification by Leveraging Web Search and Generative Models](https://mlsanity.com/paper/2310.19986/) (2023)
- [To which reference class do you belong? Measuring racial fairness of reference classes with normative modeling](https://mlsanity.com/paper/2407.19114/) (2024)
- [Biased Heritage: How Datasets Shape Models in Facial Expression Recognition](https://mlsanity.com/paper/2503.03446/) (2025)
- [A Primer on Causal and Statistical Dataset Biases for Fair and Robust Image Analysis](https://mlsanity.com/paper/2509.04295/) (2025)
- [Fairness Evaluation of Risk Estimation Models for Lung Cancer Screening](https://mlsanity.com/paper/2512.22242/) (2025)
- [FRAME: Separating sampling variation from performance disparities in medical image analysis](https://mlsanity.com/paper/2608.25981/) (2026)

## Abstract (arXiv)

An increasing number of reports raise concerns about the risk that machine learning algorithms could amplify health disparities due to biases embedded in the training data. Seyyed-Kalantari et al. find that models trained on three chest X-ray datasets yield disparities in false-positive rates (FPR) across subgroups on the 'no-finding' label (indicating the absence of disease). The models consistently yield higher FPR on subgroups known to be historically underserved, and the study concludes that the models exhibit and potentially even amplify systematic underdiagnosis. We argue that the experimental setup in the study is insufficient to study algorithmic underdiagnosis. In the absence of specific knowledge (or assumptions) about the extent and nature of the dataset bias, it is difficult to investigate model bias. Importantly, their use of test data exhibiting the same bias as the training data (due to random splitting) severely complicates the interpretation of the reported disparities.