# GIT-CXR: End-to-End Transformer for Chest X-Ray Report Generation

arXiv: https://arxiv.org/abs/2501.02598 · HTML: https://mlsanity.com/paper/2501.02598/

## Key facts

- Cited by 3 papers in the map (data through 14 Oct 2026).
- Builds on 24 of its 48 references that are in the map.
- First posted on arXiv 5 Jan 2025.

## Builds on (most cited first, 24 of 24)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [Show, Attend and Tell: Neural Image Caption Generation with Visual Attention](https://mlsanity.com/paper/1502.03044/) (2015)
- [CheXpert: A Large Chest Radiograph Dataset with Uncertainty Labels and Expert Comparison](https://mlsanity.com/paper/1901.07031/) (2019)
- [Show and Tell: A Neural Image Caption Generator](https://mlsanity.com/paper/1411.4555/) (2014)
- [Florence: A New Foundation Model for Computer Vision](https://mlsanity.com/paper/2111.11432/) (2021)
- [MIMIC-CXR-JPG, a large publicly available database of labeled chest radiographs](https://mlsanity.com/paper/1901.07042/) (2019)
- [GIT: A Generative Image-to-text Transformer for Vision and Language](https://mlsanity.com/paper/2205.14100/) (2022)
- [Generating Radiology Reports via Memory-driven Transformer](https://mlsanity.com/paper/2010.16056/) (2020)
- [CheXbert: Combining Automatic Labelers and Expert Annotations for Accurate Radiology Report Labeling Using BERT](https://mlsanity.com/paper/2004.09167/) (2020)
- [On the Automatic Generation of Medical Imaging Reports](https://mlsanity.com/paper/1711.08195/) (2017)
- [Hybrid Retrieval-Generation Reinforced Agent for Medical Image Report Generation](https://mlsanity.com/paper/1805.08298/) (2018)
- [Improving Factual Completeness and Consistency of Image-to-Text Radiology Report Generation](https://mlsanity.com/paper/2010.10042/) (2020)
- [Knowledge-driven Encode, Retrieve, Paraphrase for Medical Image Report Generation](https://mlsanity.com/paper/1903.10122/) (2019)
- [Improving Chest X-Ray Report Generation by Leveraging Warm Starting](https://mlsanity.com/paper/2201.09405/) (2022)
- [Competence-based Multimodal Curriculum Learning for Medical Report Generation](https://mlsanity.com/paper/2206.14579/) (2022)
- [GRIT: Faster and Better Image captioning Transformer Using Dual Visual Features](https://mlsanity.com/paper/2207.09666/) (2022)
- [Does the Order of Training Samples Matter? Improving Neural Data-to-Text Generation with Curriculum Learning](https://mlsanity.com/paper/2102.03554/) (2021)
- [Adversarial Generation of Natural Language](https://mlsanity.com/paper/1705.10929/) (2017)
- [Learn like a Pathologist: Curriculum Learning by Annotator Agreement for Histopathology Image Classification](https://mlsanity.com/paper/2009.13698/) (2020)
- [Automated Generation of Accurate & Fluent Medical X-ray Reports](https://mlsanity.com/paper/2108.12126/) (2021)
- [Automatic CNN-based detection of cardiac MR motion artefacts using k-space data augmentation and curriculum learning](https://mlsanity.com/paper/1810.12185/) (2018)
- [A Multi-Scale CNN and Curriculum Learning Strategy for Mammogram Classification](https://mlsanity.com/paper/1707.06978/) (2017)
- [Curriculum learning for improved femur fracture classification: scheduling data with prior knowledge and uncertainty](https://mlsanity.com/paper/2007.16102/) (2020)

## Built on by (most cited first, 3 of 3)

- [EMRRG: Efficient Fine-Tuning Pre-trained X-ray Mamba Networks for Radiology Report Generation](https://mlsanity.com/paper/2510.16776/) (2025)
- [OraPO: Oracle-educated Reinforcement Learning for Data-efficient and Factual Radiology Report Generation](https://mlsanity.com/paper/2509.18600/) (2025)
- [Multi-Modal Multi-Agent Reinforcement Learning for Radiology Report Generation](https://mlsanity.com/paper/2603.16876/) (2026)

## Abstract (arXiv)

Medical imaging is crucial for diagnosing, monitoring, and treating medical conditions. The medical reports of radiology images are the primary medium through which medical professionals attest their findings, but their writing is time consuming and requires specialized clinical expertise. The automated generation of radiography reports has thus the potential to improve and standardize patient care and significantly reduce clinicians workload. Through our work, we have designed and evaluated an end-to-end transformer-based method to generate accurate and factually complete radiology reports for X-ray images. Additionally, we are the first to introduce curriculum learning for end-to-end transformers in medical imaging and demonstrate its impact in obtaining improved performance. The experiments have been conducted using the MIMIC-CXR-JPG database, the largest available chest X-ray dataset. The results obtained are comparable with the current state-of-the-art on the natural language generation (NLG) metrics BLEU and ROUGE-L, while setting new state-of-the-art results on F1 examples-averaged, F1-macro and F1-micro metrics for clinical accuracy and on the METEOR metric widely used for NLG.