# Structural Entities Extraction and Patient Indications Incorporation for Chest X-ray Report Generation

arXiv: https://arxiv.org/abs/2405.14905 · HTML: https://mlsanity.com/paper/2405.14905/

## Key facts

- Cited by 15 papers in the map (data through 14 Oct 2026).
- Builds on 27 of its 33 references that are in the map.
- First posted on arXiv 23 May 2024.

## Builds on (most cited first, 27 of 27)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [LLaMA: Open and Efficient Foundation Language Models](https://mlsanity.com/paper/2302.13971/) (2023)
- [Billion-scale similarity search with GPUs](https://mlsanity.com/paper/1702.08734/) (2017)
- [SciBERT: A Pretrained Language Model for Scientific Text](https://mlsanity.com/paper/1903.10676/) (2019)
- [MIMIC-CXR-JPG, a large publicly available database of labeled chest radiographs](https://mlsanity.com/paper/1901.07042/) (2019)
- [Generating Radiology Reports via Memory-driven Transformer](https://mlsanity.com/paper/2010.16056/) (2020)
- [CheXbert: Combining Automatic Labelers and Expert Annotations for Accurate Radiology Report Labeling Using BERT](https://mlsanity.com/paper/2004.09167/) (2020)
- [RadGraph: Extracting Clinical Entities and Relations from Radiology Reports](https://mlsanity.com/paper/2106.14463/) (2021)
- [Cross-modal Memory Networks for Radiology Report Generation](https://mlsanity.com/paper/2204.13258/) (2022)
- [Multi-Granularity Cross-modal Alignment for Generalized Medical Visual Representation Learning](https://mlsanity.com/paper/2210.06044/) (2022)
- [Interactive and Explainable Region-guided Radiology Report Generation](https://mlsanity.com/paper/2304.08295/) (2023)
- [Knowledge-enhanced Visual-Language Pre-training on Chest Radiology Images](https://mlsanity.com/paper/2302.14042/) (2023)
- [Dynamic Graph Enhanced Contrastive Learning for Chest X-ray Report Generation](https://mlsanity.com/paper/2303.10323/) (2023)
- [METransformer: Radiology Report Generation by Transformer with Multiple Learnable Expert Tokens](https://mlsanity.com/paper/2304.02211/) (2023)
- [Improving Chest X-Ray Report Generation by Leveraging Warm Starting](https://mlsanity.com/paper/2201.09405/) (2022)
- [Improving the Factual Correctness of Radiology Report Generation with Semantic Rewards](https://mlsanity.com/paper/2210.12186/) (2022)
- [Radiology Report Generation with a Learned Knowledge Base and Multi-modal Alignment](https://mlsanity.com/paper/2112.15011/) (2021)
- [Knowledge Matters: Radiology Report Generation with General and Specific Knowledge](https://mlsanity.com/paper/2112.15009/) (2021)
- [Automatic Radiology Report Generation based on Multi-view Image Fusion and Medical Concept Enrichment](https://mlsanity.com/paper/1907.09085/) (2019)
- [PRIOR: Prototype Representation Joint Learning from Medical Images and Reports](https://mlsanity.com/paper/2307.12577/) (2023)
- [Align, Reason and Learn: Enhancing Medical Vision-and-Language Pre-training with Knowledge](https://mlsanity.com/paper/2209.07118/) (2022)
- [Towards Unifying Medical Vision-and-Language Pre-training via Soft Prompts](https://mlsanity.com/paper/2302.08958/) (2023)
- [Style-Aware Radiology Report Generation with RadGraph and Few-Shot Prompting](https://mlsanity.com/paper/2310.17811/) (2023)
- [Pragmatic Radiology Report Generation](https://mlsanity.com/paper/2311.17154/) (2023)
- [TandemNet: Distilling Knowledge from Medical Images Using Diagnostic Reports as Optional Semantic References](https://mlsanity.com/paper/1708.03070/) (2017)

## Built on by (most cited first, 15 of 15)

- [Enhanced Contrastive Learning with Multi-view Longitudinal Data for Chest X-ray Report Generation](https://mlsanity.com/paper/2502.20056/) (2025)
- [Activating Associative Disease-Aware Vision Token Memory for LLM-Based X-ray Report Generation](https://mlsanity.com/paper/2501.03458/) (2025)
- [PriorRG: Prior-Guided Contrastive Pre-training and Coarse-to-Fine Decoding for Chest X-ray Report Generation](https://mlsanity.com/paper/2508.05353/) (2025)
- [EVOKE: Elevating Chest X-ray Report Generation via Multi-View Contrastive Learning and Patient-Specific Knowledge](https://mlsanity.com/paper/2411.10224/) (2024)
- [Automated Structured Radiology Report Generation with Rich Clinical Context](https://mlsanity.com/paper/2510.00428/) (2025)
- [CheXLearner: Text-Guided Fine-Grained Representation Learning for Progression Detection](https://mlsanity.com/paper/2505.06903/) (2025)
- [DiA-gnostic VLVAE: Disentangled Alignment-Constrained Vision Language Variational AutoEncoder for Robust Radiology Reporting with Missing Modalities](https://mlsanity.com/paper/2511.05968/) (2025)
- [Multi-Modal Multi-Agent Reinforcement Learning for Radiology Report Generation](https://mlsanity.com/paper/2603.16876/) (2026)
- [Seeing Like Radiologists: Context- and Gaze-Guided Vision-Language Pretraining for Chest X-rays](https://mlsanity.com/paper/2603.26049/) (2026)
- [SURE-Med: Systematic Uncertainty Reduction for Enhanced Reliability in Medical Report Generation](https://mlsanity.com/paper/2508.01693/) (2025)
- [MMRad-22K: A Structured Multimodal Evidence Dataset for Chest X-ray Report Generation](https://mlsanity.com/paper/2602.12843/) (2026)
- [Learning to See Locally and Align Clinically with Pathology Semantics for Radiology Report Generation](https://mlsanity.com/paper/2608.00279/) (2026)
- [PDD-RRG: Posterior Diagnostic Decision for Study-level Radiology Report Generation](https://mlsanity.com/paper/2608.03055/) (2026)
- [TALON: A Temporally Aware Longitudinal Framework for Radiology Report Generation](https://mlsanity.com/paper/2609.20826/) (2026)
- [Pre-training, Reasoning, Benchmarking: X-ray Report Generation on CheXpert Plus Dataset](https://mlsanity.com/paper/2610.08813/) (2026)

## Abstract (arXiv)

The automated generation of imaging reports proves invaluable in alleviating the workload of radiologists. A clinically applicable reports generation algorithm should demonstrate its effectiveness in producing reports that accurately describe radiology findings and attend to patient-specific indications. In this paper, we introduce a novel method, Structural Entities extraction and patient indications Incorporation (SEI) for chest X-ray report generation. Specifically, we employ a structural entities extraction (SEE) approach to eliminate presentation-style vocabulary in reports and improve the quality of factual entity sequences. This reduces the noise in the following cross-modal alignment module by aligning X-ray images with factual entity sequences in reports, thereby enhancing the precision of cross-modal alignment and further aiding the model in gradient-free retrieval of similar historical cases. Subsequently, we propose a cross-modal fusion network to integrate information from X-ray images, similar historical cases, and patient-specific indications. This process allows the text decoder to attend to discriminative features of X-ray images, assimilate historical diagnostic information from similar cases, and understand the examination intention of patients. This, in turn, assists in triggering the text decoder to produce high-quality reports. Experiments conducted on MIMIC-CXR validate the superiority of SEI over state-of-the-art approaches on both natural language generation and clinical efficacy metrics.