# A clinical validation of VinDr-CXR, an AI system for detecting abnormal chest radiographs

arXiv: https://arxiv.org/abs/2104.02256 · HTML: https://mlsanity.com/paper/2104.02256/

## Key facts

- Cited by 4 papers in the map (data through 14 Oct 2026).
- Builds on 12 of its 25 references that are in the map.
- First posted on arXiv 6 Apr 2021.

## Builds on (most cited first, 12 of 12)

- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [Microsoft COCO: Common Objects in Context](https://mlsanity.com/paper/1405.0312/) (2014)
- [EfficientNet: Rethinking Model Scaling for Convolutional Neural Networks](https://mlsanity.com/paper/1905.11946/) (2019)
- [CheXpert: A Large Chest Radiograph Dataset with Uncertainty Labels and Expert Comparison](https://mlsanity.com/paper/1901.07031/) (2019)
- [EfficientDet: Scalable and Efficient Object Detection](https://mlsanity.com/paper/1911.09070/) (2019)
- [ChestX-ray8: Hospital-scale Chest X-ray Database and Benchmarks on Weakly-Supervised Classification and Localization of Common Thorax Diseases](https://mlsanity.com/paper/1705.02315/) (2017)
- [CheXNet: Radiologist-Level Pneumonia Detection on Chest X-Rays with Deep Learning](https://mlsanity.com/paper/1711.05225/) (2017)
- [VinDr-CXR: An open dataset of chest X-rays with radiologist's annotations](https://mlsanity.com/paper/2012.15029/) (2020)
- [A review of deep learning in medical imaging: Imaging traits, technology trends, case studies with progress highlights, and future promises](https://mlsanity.com/paper/2008.09104/) (2020)
- [Interpreting Chest X-rays via CNNs that Exploit Hierarchical Disease Dependencies and Uncertainty Labels](https://mlsanity.com/paper/2005.12734/) (2020)
- [CheXpedition: Investigating Generalization Challenges for Translation of Chest X-Ray Algorithms to the Clinical Setting](https://mlsanity.com/paper/2002.11379/) (2020)
- [Automatic Lung Cancer Prediction from Chest X-ray Images Using Deep Learning Approach](https://mlsanity.com/paper/1808.10858/) (2018)

## Built on by (most cited first, 4 of 4)

- [Learning to diagnose common thorax diseases on chest radiographs from radiology reports in Vietnamese](https://mlsanity.com/paper/2209.04794/) (2022)
- [Slice-level Detection of Intracranial Hemorrhage on CT Using Deep Descriptors of Adjacent Slices](https://mlsanity.com/paper/2208.03403/) (2022)
- [Improving Object Detection in Medical Image Analysis through Multiple Expert Annotators: An Empirical Investigation](https://mlsanity.com/paper/2303.16507/) (2023)
- [Evaluating the impact of an explainable machine learning system on the interobserver agreement in chest radiograph interpretation](https://mlsanity.com/paper/2304.01220/) (2023)

## Abstract (arXiv)

Computer-Aided Diagnosis (CAD) systems for chest radiographs using artificial intelligence (AI) have recently shown a great potential as a second opinion for radiologists. The performances of such systems, however, were mostly evaluated on a fixed dataset in a retrospective manner and, thus, far from the real performances in clinical practice. In this work, we demonstrate a mechanism for validating an AI-based system for detecting abnormalities on X-ray scans, VinDr-CXR, at the Phu Tho General Hospital - a provincial hospital in the North of Vietnam. The AI system was directly integrated into the Picture Archiving and Communication System (PACS) of the hospital after being trained on a fixed annotated dataset from other sources. The performance of the system was prospectively measured by matching and comparing the AI results with the radiology reports of 6,285 chest X-ray examinations extracted from the Hospital Information System (HIS) over the last two months of 2020. The normal/abnormal status of a radiology report was determined by a set of rules and served as the ground truth. Our system achieves an F1 score - the harmonic average of the recall and the precision - of 0.653 (95% CI 0.635, 0.671) for detecting any abnormalities on chest X-rays. Despite a significant drop from the in-lab performance, this result establishes a high level of confidence in applying such a system in real-life situations.