# A Systematic Review of Natural Language Processing Applied to Radiology Reports

arXiv: https://arxiv.org/abs/2102.09553 · HTML: https://mlsanity.com/paper/2102.09553/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 9 of its 132 references that are in the map.
- First posted on arXiv 18 Feb 2021.

## Builds on (most cited first, 9 of 9)

- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [Efficient Estimation of Word Representations in Vector Space](https://mlsanity.com/paper/1301.3781/) (2013)
- [Deep contextualized word representations](https://mlsanity.com/paper/1802.05365/) (2018)
- [CheXbert: Combining Automatic Labelers and Expert Annotations for Accurate Radiology Report Labeling Using BERT](https://mlsanity.com/paper/2004.09167/) (2020)
- [Optimizing the Factual Correctness of a Summary: A Study of Summarizing Radiology Reports](https://mlsanity.com/paper/1911.02541/) (2019)
- [Automated Labelling using an Attention model for Radiology reports of MRI scans (ALARM)](https://mlsanity.com/paper/2002.06588/) (2020)
- [Named Entity Recognition for Electronic Health Records: A Comparison of Rule-based and Machine Learning Approaches](https://mlsanity.com/paper/1903.03985/) (2019)
- [Surrogate-guided sampling designs for classification of rare outcomes from electronic medical records data](https://mlsanity.com/paper/1904.00412/) (2019)
- [Semi-Supervised Natural Language Approach for Fine-Grained Classification of Medical Reports](https://mlsanity.com/paper/1910.13573/) (2019)

## Abstract (arXiv)

NLP has a significant role in advancing healthcare and has been found to be key in extracting structured information from radiology reports. Understanding recent developments in NLP application to radiology is of significance but recent reviews on this are limited. This study systematically assesses recent literature in NLP applied to radiology reports. Our automated literature search yields 4,799 results using automated filtering, metadata enriching steps and citation search combined with manual review. Our analysis is based on 21 variables including radiology characteristics, NLP methodology, performance, study, and clinical application characteristics. We present a comprehensive analysis of the 164 publications retrieved with each categorised into one of 6 clinical application categories. Deep learning use increases but conventional machine learning approaches are still prevalent. Deep learning remains challenged when data is scarce and there is little evidence of adoption into clinical practice. Despite 17% of studies reporting greater than 0.85 F1 scores, it is hard to comparatively evaluate these approaches given that most of them use different datasets. Only 14 studies made their data and 15 their code available with 10 externally validating results. Automated understanding of clinical narratives of the radiology reports has the potential to enhance the healthcare process but reproducibility and explainability of models are important if the domain is to move applications into clinical use. More could be done to share code enabling validation of methods on different institutional data and to reduce heterogeneity in reporting of study properties allowing inter-study comparisons. Our results have significance for researchers providing a systematic synthesis of existing work to build on, identify gaps, opportunities for collaboration and avoid duplication.