# Extraction and Analysis of Clinically Important Follow-up Recommendations in a Large Radiology Dataset

arXiv: https://arxiv.org/abs/1905.05877 · HTML: https://mlsanity.com/paper/1905.05877/

## Key facts

- Cited by 5 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 14 May 2019.

## Built on by (most cited first, 5 of 5)

- [Event-based clinical findings extraction from radiology reports with pre-trained language model](https://mlsanity.com/paper/2112.13512/) (2021)
- [Identifying Imaging Follow-Up in Radiology Reports: A Comparative Analysis of Traditional ML and LLM Approaches](https://mlsanity.com/paper/2511.11867/) (2025)
- [Checkup2Action: A Multimodal Clinical Check-up Report Dataset for Patient-Oriented Action Card Generation](https://mlsanity.com/paper/2605.11533/) (2026)
- [Automated Identification of Incidentalomas Requiring Follow-Up: A Multi-Anatomy Evaluation of LLM-Based and Supervised Approaches](https://mlsanity.com/paper/2512.05537/) (2025)
- [Clinical Intent Extraction: A FHIR-Aligned Representation and the CIRCA Benchmark](https://mlsanity.com/paper/2609.29479/) (2026)

## Abstract (arXiv)

Communication of follow-up recommendations when abnormalities are identified on imaging studies is prone to error. In this paper, we present a natural language processing approach based on deep learning to automatically identify clinically important recommendations in radiology reports. Our approach first identifies the recommendation sentences and then extracts reason, test, and time frame of the identified recommendations. To train our extraction models, we created a corpus of 567 radiology reports annotated for recommendation information. Our extraction models achieved 0.92 f-score for recommendation sentence, 0.65 f-score for reason, 0.73 f-score for test, and 0.84 f-score for time frame. We applied the extraction models to a set of over 3.3 million radiology reports and analyzed the adherence of follow-up recommendations.