# CheXpedition: Investigating Generalization Challenges for Translation of Chest X-Ray Algorithms to the Clinical Setting

arXiv: https://arxiv.org/abs/2002.11379 · HTML: https://mlsanity.com/paper/2002.11379/

## Key facts

- Cited by 18 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 26 Feb 2020.

## Built on by (most cited first, 18 of 18)

- [CheXtransfer: Performance and Parameter Efficiency of ImageNet Models for Chest X-Ray Interpretation](https://mlsanity.com/paper/2101.06871/) (2021)
- [Supervised Transfer Learning at Scale for Medical Imaging](https://mlsanity.com/paper/2101.05913/) (2021)
- [No True State-of-the-Art? OOD Detection Methods are Inconsistent across Datasets](https://mlsanity.com/paper/2109.05554/) (2021)
- [VisualCheXbert: Addressing the Discrepancy Between Radiology Report Labels and Image Labels](https://mlsanity.com/paper/2102.11467/) (2021)
- [PediCXR: An open, large-scale chest radiograph dataset for interpretation of common thoracic diseases in children](https://mlsanity.com/paper/2203.10612/) (2022)
- [Probabilistic Integration of Object Level Annotations in Chest X-ray Classification](https://mlsanity.com/paper/2210.06980/) (2022)
- [An Accurate and Explainable Deep Learning System Improves Interobserver Agreement in the Interpretation of Chest Radiograph](https://mlsanity.com/paper/2208.03545/) (2022)
- [Key Technology Considerations in Developing and Deploying Machine Learning Models in Clinical Radiology Practice](https://mlsanity.com/paper/2102.01979/) (2021)
- [Three Applications of Conformal Prediction for Rating Breast Density in Mammography](https://mlsanity.com/paper/2206.12008/) (2022)
- [CheXternal: Generalization of Deep Learning Models for Chest X-ray Interpretation to Photos of Chest X-rays and External Clinical Settings](https://mlsanity.com/paper/2102.08660/) (2021)
- [Learning to diagnose common thorax diseases on chest radiographs from radiology reports in Vietnamese](https://mlsanity.com/paper/2209.04794/) (2022)
- [A clinical validation of VinDr-CXR, an AI system for detecting abnormal chest radiographs](https://mlsanity.com/paper/2104.02256/) (2021)
- [Meta-repository of screening mammography classifiers](https://mlsanity.com/paper/2108.04800/) (2021)
- [CheXseen: Unseen Disease Detection for Deep Learning Interpretation of Chest X-rays](https://mlsanity.com/paper/2103.04590/) (2021)
- [Performance Deterioration of Deep Learning Models after Clinical Deployment: A Case Study with Auto-segmentation for Definitive Prostate Cancer Radiotherapy](https://mlsanity.com/paper/2210.05673/) (2022)
- [CheXbreak: Misclassification Identification for Deep Learning Models Interpreting Chest X-rays](https://mlsanity.com/paper/2103.09957/) (2021)
- [Learning Generalized Medical Image Representations through Image-Graph Contrastive Pretraining](https://mlsanity.com/paper/2405.09594/) (2024)
- [Deep learning for detecting pulmonary tuberculosis via chest radiography: an international study across 10 countries](https://mlsanity.com/paper/2105.07540/) (2021)

## Abstract (arXiv)

Although there have been several recent advances in the application of deep learning algorithms to chest x-ray interpretation, we identify three major challenges for the translation of chest x-ray algorithms to the clinical setting. We examine the performance of the top 10 performing models on the CheXpert challenge leaderboard on three tasks: (1) TB detection, (2) pathology detection on photos of chest x-rays, and (3) pathology detection on data from an external institution. First, we find that the top 10 chest x-ray models on the CheXpert competition achieve an average AUC of 0.851 on the task of detecting TB on two public TB datasets without fine-tuning or including the TB labels in training data. Second, we find that the average performance of the models on photos of x-rays (AUC = 0.916) is similar to their performance on the original chest x-ray images (AUC = 0.924). Third, we find that the models tested on an external dataset either perform comparably to or exceed the average performance of radiologists. We believe that our investigation will inform rapid translation of deep learning algorithms to safe and effective clinical decision support tools that can be validated prospectively with large impact studies and clinical trials.