# Controllable Chest X-Ray Report Generation from Longitudinal Representations

arXiv: https://arxiv.org/abs/2310.05881 · HTML: https://mlsanity.com/paper/2310.05881/

## Key facts

- Cited by 23 papers in the map (data through 14 Oct 2026).
- Builds on 27 of its 49 references that are in the map.
- First posted on arXiv 9 Oct 2023.

## Builds on (most cited first, 27 of 27)

- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [GPT-4 Technical Report](https://mlsanity.com/paper/2303.08774/) (2023)
- [LLaMA: Open and Efficient Foundation Language Models](https://mlsanity.com/paper/2302.13971/) (2023)
- [Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Networks](https://mlsanity.com/paper/1506.01497/) (2015)
- [Graph Attention Networks](https://mlsanity.com/paper/1710.10903/) (2017)
- [Densely Connected Convolutional Networks](https://mlsanity.com/paper/1608.06993/) (2016)
- [Feature Pyramid Networks for Object Detection](https://mlsanity.com/paper/1612.03144/) (2016)
- [Bottom-Up and Top-Down Attention for Image Captioning and Visual Question Answering](https://mlsanity.com/paper/1707.07998/) (2017)
- [Oscar: Object-Semantics Aligned Pre-training for Vision-Language Tasks](https://mlsanity.com/paper/2004.06165/) (2020)
- [MIMIC-CXR-JPG, a large publicly available database of labeled chest radiographs](https://mlsanity.com/paper/1901.07042/) (2019)
- [Generating Radiology Reports via Memory-driven Transformer](https://mlsanity.com/paper/2010.16056/) (2020)
- [Meshed-Memory Transformer for Image Captioning](https://mlsanity.com/paper/1912.08226/) (2019)
- [CheXbert: Combining Automatic Labelers and Expert Annotations for Accurate Radiology Report Labeling Using BERT](https://mlsanity.com/paper/2004.09167/) (2020)
- [Learning to Exploit Temporal Structure for Biomedical Vision-Language Processing](https://mlsanity.com/paper/2301.04558/) (2023)
- [Cross-modal Memory Networks for Radiology Report Generation](https://mlsanity.com/paper/2204.13258/) (2022)
- [Exploring and Distilling Posterior and Prior Knowledge for Radiology Report Generation](https://mlsanity.com/paper/2106.06963/) (2021)
- [Improving Factual Completeness and Consistency of Image-to-Text Radiology Report Generation](https://mlsanity.com/paper/2010.10042/) (2020)
- [Interactive and Explainable Region-guided Radiology Report Generation](https://mlsanity.com/paper/2304.08295/) (2023)
- [Clinically Accurate Chest X-Ray Report Generation](https://mlsanity.com/paper/1904.02633/) (2019)
- [Benchmarking Detection Transfer Learning with Vision Transformers](https://mlsanity.com/paper/2111.11429/) (2021)
- [Chest ImaGenome Dataset for Clinical Reasoning](https://mlsanity.com/paper/2108.00316/) (2021)
- [Knowledge Matters: Radiology Report Generation with General and Specific Knowledge](https://mlsanity.com/paper/2112.15009/) (2021)
- [Improving Radiology Report Generation Systems by Removing Hallucinated References to Non-existent Priors](https://mlsanity.com/paper/2210.06340/) (2022)
- [CheXRelNet: An Anatomy-Aware Model for Tracking Longitudinal Relationships between Chest X-Rays](https://mlsanity.com/paper/2208.03873/) (2022)
- [Finding-Aware Anatomical Tokens for Chest X-Ray Automated Reporting](https://mlsanity.com/paper/2308.15961/) (2023)

## Built on by (most cited first, 23 of 23)

- [MAIRA-2: Grounded Radiology Report Generation](https://mlsanity.com/paper/2406.04449/) (2024)
- [Automated Radiology Report Generation: A Review of Recent Advances](https://mlsanity.com/paper/2405.10842/) (2024)
- [HERGen: Elevating Radiology Report Generation with Longitudinal Data](https://mlsanity.com/paper/2407.15158/) (2024)
- [Libra: Leveraging Temporal Images for Biomedical Radiology Analysis](https://mlsanity.com/paper/2411.19378/) (2024)
- [LLM-RG4: Flexible and Factual Radiology Report Generation across Diverse Input Contexts](https://mlsanity.com/paper/2412.12001/) (2024)
- [RADAR: Enhancing Radiology Report Generation with Supplementary Knowledge Injection](https://mlsanity.com/paper/2505.14318/) (2025)
- [Scaling medical imaging report generation with multimodal reinforcement learning](https://mlsanity.com/paper/2601.17151/) (2026)
- [DDaTR: Dynamic Difference-aware Temporal Residual Network for Longitudinal Radiology Report Generation](https://mlsanity.com/paper/2505.03401/) (2025)
- [A Survey of Deep Learning-based Radiology Report Generation Using Multimodal Data](https://mlsanity.com/paper/2405.12833/) (2024)
- [TiBiX: Leveraging Temporal Information for Bidirectional X-ray and Report Generation](https://mlsanity.com/paper/2403.13343/) (2024)
- [Grounding Chest X-Ray Visual Question Answering with Generated Radiology Reports](https://mlsanity.com/paper/2505.16624/) (2025)
- [A Review of Longitudinal Radiology Report Generation: Dataset Composition, Methods, and Performance Evaluation](https://mlsanity.com/paper/2510.12444/) (2025)
- [ICON: Improving Inter-Report Consistency in Radiology Report Generation via Lesion-aware Mixup Augmentation](https://mlsanity.com/paper/2402.12844/) (2024)
- [Automated Structured Radiology Report Generation with Rich Clinical Context](https://mlsanity.com/paper/2510.00428/) (2025)
- [Multimodal Disease Progression Modeling via Spatiotemporal Disentanglement and Multiscale Alignment](https://mlsanity.com/paper/2510.11112/) (2025)
- [A Vision-language Framework for Comparative Reasoning in Radiology](https://mlsanity.com/paper/2606.06407/) (2026)
- [Aligning Characteristic Descriptors with Images for Human-Expert-like Explainability](https://mlsanity.com/paper/2411.04008/) (2024)
- [Standardizing Longitudinal Radiology Report Evaluation via Large Language Model Annotation](https://mlsanity.com/paper/2601.16753/) (2026)
- [PDD-RRG: Posterior Diagnostic Decision for Study-level Radiology Report Generation](https://mlsanity.com/paper/2608.03055/) (2026)
- [ALTER: Modeling Longitudinal Changes via Regional Differencing for 3D CT Report Generation](https://mlsanity.com/paper/2608.05615/) (2026)
- [CheXGround: Anatomical Region Tokens for Grounded Longitudinal Chest X-ray Interpretation](https://mlsanity.com/paper/2608.30758/) (2026)
- [BrainDiff: Longitudinal Report Generation for Multimodal Brain MRI](https://mlsanity.com/paper/2609.00593/) (2026)
- [SHIFT-M3: Pre-fusion Alignment-based Consistency Screening for Multimodal ECG Record Integrity](https://mlsanity.com/paper/2609.13874/) (2026)

## Abstract (arXiv)

Radiology reports are detailed text descriptions of the content of medical scans. Each report describes the presence/absence and location of relevant clinical findings, commonly including comparison with prior exams of the same patient to describe how they evolved. Radiology reporting is a time-consuming process, and scan results are often subject to delays. One strategy to speed up reporting is to integrate automated reporting systems, however clinical deployment requires high accuracy and interpretability. Previous approaches to automated radiology reporting generally do not provide the prior study as input, precluding comparison which is required for clinical accuracy in some types of scans, and offer only unreliable methods of interpretability. Therefore, leveraging an existing visual input format of anatomical tokens, we introduce two novel aspects: (1) longitudinal representation learning -- we input the prior scan as an additional input, proposing a method to align, concatenate and fuse the current and prior visual information into a joint longitudinal representation which can be provided to the multimodal report generation model; (2) sentence-anatomy dropout -- a training strategy for controllability in which the report generator model is trained to predict only sentences from the original report which correspond to the subset of anatomical regions given as input. We show through in-depth experiments on the MIMIC-CXR dataset how the proposed approach achieves state-of-the-art results while enabling anatomy-wise controllable report generation.