# An Accurate and Explainable Deep Learning System Improves Interobserver Agreement in the Interpretation of Chest Radiograph

arXiv: https://arxiv.org/abs/2208.03545 · HTML: https://mlsanity.com/paper/2208.03545/

## Key facts

- Cited by 7 papers in the map (data through 14 Oct 2026).
- Builds on 33 of its 84 references that are in the map.
- First posted on arXiv 6 Aug 2022.

## Builds on (most cited first, 30 of 33)

- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [Microsoft COCO: Common Objects in Context](https://mlsanity.com/paper/1405.0312/) (2014)
- [EfficientNet: Rethinking Model Scaling for Convolutional Neural Networks](https://mlsanity.com/paper/1905.11946/) (2019)
- [CheXpert: A Large Chest Radiograph Dataset with Uncertainty Labels and Expert Comparison](https://mlsanity.com/paper/1901.07031/) (2019)
- [A Survey on Deep Learning in Medical Image Analysis](https://mlsanity.com/paper/1702.05747/) (2017)
- [EfficientDet: Scalable and Efficient Object Detection](https://mlsanity.com/paper/1911.09070/) (2019)
- [ChestX-ray8: Hospital-scale Chest X-ray Database and Benchmarks on Weakly-Supervised Classification and Localization of Common Thorax Diseases](https://mlsanity.com/paper/1705.02315/) (2017)
- [CheXNet: Radiologist-Level Pneumonia Detection on Chest X-Rays with Deep Learning](https://mlsanity.com/paper/1711.05225/) (2017)
- [Confounding variables can degrade generalization performance of radiological deep learning models](https://mlsanity.com/paper/1807.00431/) (2018)
- [CheXbert: Combining Automatic Labelers and Expert Annotations for Accurate Radiology Report Labeling Using BERT](https://mlsanity.com/paper/2004.09167/) (2020)
- [PadChest: A large chest x-ray image dataset with multi-label annotated reports](https://mlsanity.com/paper/1901.07441/) (2019)
- [VinDr-CXR: An open dataset of chest X-rays with radiologist's annotations](https://mlsanity.com/paper/2012.15029/) (2020)
- [A deep learning system for differential diagnosis of skin diseases](https://mlsanity.com/paper/1909.05382/) (2019)
- [Deep Neural Networks Improve Radiologists' Performance in Breast Cancer Screening](https://mlsanity.com/paper/1903.08297/) (2019)
- [VinDr-Mammo: A large-scale benchmark dataset for computer-aided diagnosis in full-field digital mammography](https://mlsanity.com/paper/2203.11205/) (2022)
- [On the limits of cross-domain generalization in automated X-ray prediction](https://mlsanity.com/paper/2002.02497/) (2020)
- [Interpreting Chest X-rays via CNNs that Exploit Hierarchical Disease Dependencies and Uncertainty Labels](https://mlsanity.com/paper/2005.12734/) (2020)
- [Reliable Tuberculosis Detection using Chest X-ray with Deep Learning, Segmentation and Visualization](https://mlsanity.com/paper/2007.14895/) (2020)
- [CheXplain: Enabling Physicians to Explore and UnderstandData-Driven, AI-Enabled Medical Imaging Analysis](https://mlsanity.com/paper/2001.05149/) (2020)
- [Exploring large scale public medical image datasets](https://mlsanity.com/paper/1907.12720/) (2019)
- [VinDr-RibCXR: A Benchmark Dataset for Automatic Segmentation and Labeling of Individual Ribs on Chest X-rays](https://mlsanity.com/paper/2107.01327/) (2021)
- [VinDr-SpineXR: A deep learning framework for spinal lesions detection and classification from radiographs](https://mlsanity.com/paper/2106.12930/) (2021)
- [PediCXR: An open, large-scale chest radiograph dataset for interpretation of common thoracic diseases in children](https://mlsanity.com/paper/2203.10612/) (2022)
- [CheXpedition: Investigating Generalization Challenges for Translation of Chest X-Ray Algorithms to the Clinical Setting](https://mlsanity.com/paper/2002.11379/) (2020)
- [A novel multi-view deep learning approach for BI-RADS and density assessment of mammograms](https://mlsanity.com/paper/2112.04490/) (2021)
- [Learning from Multiple Expert Annotators for Enhancing Anomaly Detection in Medical Image Analysis](https://mlsanity.com/paper/2203.10611/) (2022)
- [DICOM Imaging Router: An Open Deep Learning Framework for Classification of Body Parts from DICOM X-ray Scans](https://mlsanity.com/paper/2108.06490/) (2021)
- [Phase Recognition in Contrast-Enhanced CT Scans based on Deep Learning and Random Sampling](https://mlsanity.com/paper/2203.11206/) (2022)
- [Advance Warning Methodologies for COVID-19 using Chest X-Ray Images](https://mlsanity.com/paper/2006.05332/) (2020)
- [Artificial Intelligence applied to chest X-Ray images for the automatic detection of COVID-19. A thoughtful evaluation approach](https://mlsanity.com/paper/2011.14259/) (2020)

## Built on by (most cited first, 7 of 7)

- [MedRAX: Medical Reasoning Agent for Chest X-ray](https://mlsanity.com/paper/2502.02673/) (2025)
- [Read Like a Radiologist: Efficient Vision-Language Model for 3D Medical Imaging Interpretation](https://mlsanity.com/paper/2412.13558/) (2024)
- [Label Convergence: Defining an Upper Performance Bound in Object Recognition through Contradictory Annotations](https://mlsanity.com/paper/2409.09412/) (2024)
- [Learning to diagnose common thorax diseases on chest radiographs from radiology reports in Vietnamese](https://mlsanity.com/paper/2209.04794/) (2022)
- [Improving Object Detection in Medical Image Analysis through Multiple Expert Annotators: An Empirical Investigation](https://mlsanity.com/paper/2303.16507/) (2023)
- [Evaluating the impact of an explainable machine learning system on the interobserver agreement in chest radiograph interpretation](https://mlsanity.com/paper/2304.01220/) (2023)
- [Example-based Robust Abnormality Detection with Minimal Annotations using Exemplar Med-DETR](https://mlsanity.com/paper/2608.24281/) (2026)

## Abstract (arXiv)

Recent artificial intelligence (AI) algorithms have achieved radiologist-level performance on various medical classification tasks. However, only a few studies addressed the localization of abnormal findings from CXR scans, which is essential in explaining the image-level classification to radiologists. We introduce in this paper an explainable deep learning system called VinDr-CXR that can classify a CXR scan into multiple thoracic diseases and, at the same time, localize most types of critical findings on the image. VinDr-CXR was trained on 51,485 CXR scans with radiologist-provided bounding box annotations. It demonstrated a comparable performance to experienced radiologists in classifying 6 common thoracic diseases on a retrospective validation set of 3,000 CXR scans, with a mean area under the receiver operating characteristic curve (AUROC) of 0.967 (95% confidence interval [CI]: 0.958-0.975). The VinDr-CXR was also externally validated in independent patient cohorts and showed its robustness. For the localization task with 14 types of lesions, our free-response receiver operating characteristic (FROC) analysis showed that the VinDr-CXR achieved a sensitivity of 80.2% at the rate of 1.0 false-positive lesion identified per scan. A prospective study was also conducted to measure the clinical impact of the VinDr-CXR in assisting six experienced radiologists. The results indicated that the proposed system, when used as a diagnosis supporting tool, significantly improved the agreement between radiologists themselves with an increase of 1.5% in mean Fleiss' Kappa. We also observed that, after the radiologists consulted VinDr-CXR's suggestions, the agreement between each of them and the system was remarkably increased by 3.3% in mean Cohen's Kappa.