# Caveats in Generating Medical Imaging Labels from Radiology Reports

arXiv: https://arxiv.org/abs/1905.02283 · HTML: https://mlsanity.com/paper/1905.02283/

## Key facts

- Cited by 9 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 6 May 2019.

## Built on by (most cited first, 9 of 9)

- [Deep Learning for Chest X-ray Analysis: A Survey](https://mlsanity.com/paper/2103.08700/) (2021)
- [VisualCheXbert: Addressing the Discrepancy Between Radiology Report Labels and Image Labels](https://mlsanity.com/paper/2102.11467/) (2021)
- [XReal: Realistic Anatomy and Pathology-Aware X-ray Generation via Controllable Diffusion Model](https://mlsanity.com/paper/2403.09240/) (2024)
- [Key Technology Considerations in Developing and Deploying Machine Learning Models in Clinical Radiology Practice](https://mlsanity.com/paper/2102.01979/) (2021)
- [Did You Get What You Paid For? Rethinking Annotation Cost of Deep Learning Based Computer Aided Detection in Chest Radiographs](https://mlsanity.com/paper/2209.15314/) (2022)
- [Improving Classification Model Performance on Chest X-Rays through Lung Segmentation](https://mlsanity.com/paper/2202.10971/) (2022)
- [Deep Learning Models for Calculation of Cardiothoracic Ratio from Chest Radiographs for Assisted Diagnosis of Cardiomegaly](https://mlsanity.com/paper/2101.07606/) (2021)
- [Tips and Tricks to Improve CNN-based Chest X-ray Diagnosis: A Survey](https://mlsanity.com/paper/2106.00997/) (2021)
- [Reconfigurable Radiology Labels Without Relabeling](https://mlsanity.com/paper/2607.06597/) (2026)

## Abstract (arXiv)

Acquiring high-quality annotations in medical imaging is usually a costly process. Automatic label extraction with natural language processing (NLP) has emerged as a promising workaround to bypass the need of expert annotation. Despite the convenience, the limitation of such an approximation has not been carefully examined and is not well understood. With a challenging set of 1,000 chest X-ray studies and their corresponding radiology reports, we show that there exists a surprisingly large discrepancy between what radiologists visually perceive and what they clinically report. Furthermore, with inherently flawed report as ground truth, the state-of-the-art medical NLP fails to produce high-fidelity labels.