# CheXternal: Generalization of Deep Learning Models for Chest X-ray Interpretation to Photos of Chest X-rays and External Clinical Settings

arXiv: https://arxiv.org/abs/2102.08660 · HTML: https://mlsanity.com/paper/2102.08660/

## Key facts

- Cited by 5 papers in the map (data through 14 Oct 2026).
- Builds on 9 of its 35 references that are in the map.
- First posted on arXiv 17 Feb 2021.

## Builds on (most cited first, 9 of 9)

- [Benchmarking Neural Network Robustness to Common Corruptions and Perturbations](https://mlsanity.com/paper/1903.12261/) (2019)
- [Adversarial examples in the physical world](https://mlsanity.com/paper/1607.02533/) (2016)
- [ImageNet-trained CNNs are biased towards texture; increasing shape bias improves accuracy and robustness](https://mlsanity.com/paper/1811.12231/) (2018)
- [CheXpert: A Large Chest Radiograph Dataset with Uncertainty Labels and Expert Comparison](https://mlsanity.com/paper/1901.07031/) (2019)
- [Confounding variables can degrade generalization performance of radiological deep learning models](https://mlsanity.com/paper/1807.00431/) (2018)
- [Adversarially Robust Generalization Requires More Data](https://mlsanity.com/paper/1804.11285/) (2018)
- [A Study and Comparison of Human and Deep Learning Recognition Performance Under Visual Distortions](https://mlsanity.com/paper/1705.02498/) (2017)
- [CheXpedition: Investigating Generalization Challenges for Translation of Chest X-Ray Algorithms to the Clinical Setting](https://mlsanity.com/paper/2002.11379/) (2020)
- [CheXphoto: 10,000+ Photos and Transformations of Chest X-rays for Benchmarking Deep Learning Robustness](https://mlsanity.com/paper/2007.06199/) (2020)

## Built on by (most cited first, 5 of 5)

- [Knowledge-enhanced Visual-Language Pre-training on Chest Radiology Images](https://mlsanity.com/paper/2302.14042/) (2023)
- [CARZero: Cross-Attention Alignment for Radiology Zero-Shot Classification](https://mlsanity.com/paper/2402.17417/) (2024)
- [Human Expertise in Algorithmic Prediction](https://mlsanity.com/paper/2402.00793/) (2024)
- [Image Projective Transformation Rectification with Synthetic Data for Smartphone-captured Chest X-ray Photos Classification](https://mlsanity.com/paper/2210.05954/) (2022)
- [STAMP: Predicting Out-of-Distribution Generalization without Target Data](https://mlsanity.com/paper/2609.32672/) (2026)

## Abstract (arXiv)

Recent advances in training deep learning models have demonstrated the potential to provide accurate chest X-ray interpretation and increase access to radiology expertise. However, poor generalization due to data distribution shifts in clinical settings is a key barrier to implementation. In this study, we measured the diagnostic performance for 8 different chest X-ray models when applied to (1) smartphone photos of chest X-rays and (2) external datasets without any finetuning. All models were developed by different groups and submitted to the CheXpert challenge, and re-applied to test datasets without further tuning. We found that (1) on photos of chest X-rays, all 8 models experienced a statistically significant drop in task performance, but only 3 performed significantly worse than radiologists on average, and (2) on the external set, none of the models performed statistically significantly worse than radiologists, and five models performed statistically significantly better than radiologists. Our results demonstrate that some chest X-ray models, under clinically relevant distribution shifts, were comparable to radiologists while other models were not. Future work should investigate aspects of model training procedures and dataset collection that influence generalization in the presence of data distribution shifts.