# Clinical Context-aware Radiology Report Generation from Medical Images using Transformers

arXiv: https://arxiv.org/abs/2408.11344 · HTML: https://mlsanity.com/paper/2408.11344/

## Key facts

- Cited by 2 papers in the map (data through 14 Oct 2026).
- Builds on 11 of its 26 references that are in the map.
- First posted on arXiv 21 Aug 2024.

## Builds on (most cited first, 11 of 11)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [PyTorch: An Imperative Style, High-Performance Deep Learning Library](https://mlsanity.com/paper/1912.01703/) (2019)
- [Sequence to Sequence Learning with Neural Networks](https://mlsanity.com/paper/1409.3215/) (2014)
- [Show and Tell: A Neural Image Caption Generator](https://mlsanity.com/paper/1411.4555/) (2014)
- [On the Automatic Generation of Medical Imaging Reports](https://mlsanity.com/paper/1711.08195/) (2017)
- [Hybrid Retrieval-Generation Reinforced Agent for Medical Image Report Generation](https://mlsanity.com/paper/1805.08298/) (2018)
- [Clinically Accurate Chest X-Ray Report Generation](https://mlsanity.com/paper/1904.02633/) (2019)
- [Knowledge-driven Encode, Retrieve, Paraphrase for Medical Image Report Generation](https://mlsanity.com/paper/1903.10122/) (2019)
- [Learning to Read Chest X-Rays: Recurrent Neural Cascade Model for Automated Image Annotation](https://mlsanity.com/paper/1603.08486/) (2016)
- [MDNet: A Semantically and Visually Interpretable Medical Image Diagnosis Network](https://mlsanity.com/paper/1707.02485/) (2017)

## Built on by (most cited first, 2 of 2)

- [Designing a Robust Radiology Report Generation System](https://mlsanity.com/paper/2411.01153/) (2024)
- [AI-Driven Radiology Report Generation for Traumatic Brain Injuries](https://mlsanity.com/paper/2510.08498/) (2025)

## Abstract (arXiv)

Recent developments in the field of Natural Language Processing, especially language models such as the transformer have brought state-of-the-art results in language understanding and language generation. In this work, we investigate the use of the transformer model for radiology report generation from chest X-rays. We also highlight limitations in evaluating radiology report generation using only the standard language generation metrics. We then applied a transformer based radiology report generation architecture, and also compare the performance of a transformer based decoder with the recurrence based decoder. Experiments were performed using the IU-CXR dataset, showing superior results to its LSTM counterpart and being significantly faster. Finally, we identify the need of evaluating radiology report generation system using both language generation metrics and classification metrics, which helps to provide robust measure of generated reports in terms of their coherence and diagnostic value.