mlsanityData to 7 Oct 2026

Improving Chest X-Ray Report Generation by Leveraging Warm Starting

arXiv 2201.09405 ↗First posted 24 Jan 2022cs.CVcited by 93

Not on the map. Why?

Abstract (arXiv)

Automatically generating a report from a patient's Chest X-Rays (CXRs) is a promising solution to reducing clinical workload and improving patient care. However, current CXR report generators -- which are predominantly encoder-to-decoder models -- lack the diagnostic accuracy to be deployed in a clinical setting. To improve CXR report generation, we investigate warm starting the encoder and decoder with recent open-source computer vision and natural language processing checkpoints, such as the Vision Transformer (ViT) and PubMedBERT. To this end, each checkpoint is evaluated on the MIMIC-CXR and IU X-Ray datasets. Our experimental investigation demonstrates that the Convolutional vision Transformer (CvT) ImageNet-21K and the Distilled Generative Pre-trained Transformer 2 (DistilGPT2) checkpoints are best for warm starting the encoder and decoder, respectively. Compared to the state-of-the-art (M2 Transformer Progressive), CvT2DistilGPT2 attained an improvement of 8.3% for CE F-1, 1.8% for BLEU-4, 1.6% for ROUGE-L, and 1.0% for METEOR. The reports generated by CvT2DistilGPT2 have a higher similarity to radiologist reports than previous approaches. This indicates that leveraging warm starting improves CXR report generation. Code and checkpoints for CvT2DistilGPT2 are available at https://github.com/aehrc/cvt2distilgpt2.

Key facts. Cited by 93 papers in the map (data through 7 Oct 2026). Builds on 45 of its 84 references that are in the map. First posted on arXiv 24 Jan 2022.

Builds on

45 of 84 references.

References · 45

  1. 2017Attention Is All You Needcited by 59,932
  2. 2015Deep Residual Learning for Image Recognitioncited by 51,809
  3. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,204
  4. 2020An Image is Worth 16x16 Words: Transformers for Image Recognition at Scalecited by 29,915
  5. 2014ImageNet Large Scale Visual Recognition Challengecited by 8,441
  6. 2016Densely Connected Convolutional Networkscited by 6,691
  7. 2019EfficientNet: Rethinking Model Scaling for Convolutional Neural Networkscited by 6,415
  8. 2019BART: Denoising Sequence-to-Sequence Pre-training for Natural Language Generation, Translation, and Comprehensioncited by 5,874
  9. 2020Training data-efficient image transformers & distillation through attentioncited by 4,560
  10. 2019DistilBERT, a distilled version of BERT: smaller, faster, cheaper and lightercited by 4,243
  11. 2021BEiT: BERT Pre-Training of Image Transformerscited by 2,360
  12. 2014CIDEr: Consensus-based Image Description Evaluationcited by 2,027
33 more
  1. 2019BioBERT: a pre-trained biomedical language representation model for biomedical text miningcited by 1,935
  2. 2019CheXpert: A Large Chest Radiograph Dataset with Uncertainty Labels and Expert Comparisoncited by 1,410
  3. 2019SciBERT: A Pretrained Language Model for Scientific Textcited by 1,404
  4. 2019A Comprehensive Survey on Transfer Learningcited by 1,154
  5. 2015Aligning Books and Movies: Towards Story-like Visual Explanations by Watching Movies and Reading Bookscited by 1,041
  6. 2020Domain-Specific Language Model Pretraining for Biomedical Natural Language Processingcited by 996
  7. 2019Publicly Available Clinical BERT Embeddingscited by 995
  8. 2021CvT: Introducing Convolutions to Vision Transformerscited by 946
  9. 2019Big Transfer (BiT): General Visual Representation Learningcited by 766
  10. 2019MIMIC-CXR-JPG, a large publicly available database of labeled chest radiographscited by 590
  11. 2016Self-critical Sequence Training for Image Captioningcited by 544
  12. 2017CheXNet: Radiologist-Level Pneumonia Detection on Chest X-Rays with Deep Learningcited by 459
  13. 2021Intriguing Properties of Vision Transformerscited by 446
  14. 2021Early Convolutions Help Transformers See Bettercited by 395
  15. 2020Generating Radiology Reports via Memory-driven Transformercited by 363
  16. 2019Meshed-Memory Transformer for Image Captioningcited by 343
  17. 2021XCiT: Cross-Covariance Image Transformerscited by 335
  18. 2020CheXbert: Combining Automatic Labelers and Expert Annotations for Accurate Radiology Report Labeling Using BERTcited by 329
  19. 2019Transfer Learning in Biomedical Natural Language Processing: An Evaluation of BERT and ELMo on Ten Benchmarking Datasetscited by 298
  20. 2017On the Automatic Generation of Medical Imaging Reportscited by 215
  21. 2019Leveraging Pre-trained Checkpoints for Sequence Generation Taskscited by 182
  22. 2021Exploring and Distilling Posterior and Prior Knowledge for Radiology Report Generationcited by 160
  23. 2020When Radiology Report Generation Meets Knowledge Graphcited by 144
  24. 2018Hybrid Retrieval-Generation Reinforced Agent for Medical Image Report Generationcited by 141
  25. 2018TieNet: Text-Image Embedding Network for Common Thorax Disease Classification and Reporting in Chest X-rayscited by 139
  26. 2019Clinically Accurate Chest X-Ray Report Generationcited by 138
  27. 2021Geometry of the Loss Landscape in Overparameterized Neural Networks: Symmetries and Invariancescited by 125
  28. 2021Contrastive Attention for Automatic Chest X-ray Report Generationcited by 71
  29. 2020Show, Describe and Conclude: On Exploiting the Structure Information of Chest X-Ray Reportscited by 58
  30. 2021CheXtransfer: Performance and Parameter Efficiency of ImageNet Models for Chest X-Ray Interpretationcited by 57
  31. 2021Progressive Transformer-Based Generation of Radiology Reportscited by 43
  32. 2018Input Combination Strategies for Multi-Source Transformer Decodercited by 22
  33. 2021Diagnostic Captioning: A Surveycited by 6

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2023: 13; 2024: 33; 2025: 29; 2026: 18.

93 papers.

Sort:

By area:

  1. 2023Towards Generalist Biomedical AIcited by 286
  2. 2023Conformal Language Modelingcited by 163
  3. 2023A Survey of Large Language Models for Healthcare: from Data, Technology, and Applications to Accountability and Ethicscited by 162
  4. 2023Interactive and Explainable Region-guided Radiology Report Generationcited by 139
  5. 2023ChatCAD: Interactive Computer-Aided Diagnosis on Medical Image using Large Language Modelscited by 116
  6. 2023PromptMRG: Diagnosis-Driven Prompts for Medical Report Generationcited by 92
  7. 2023MAIRA-1: A specialised large multimodal model for radiology report generationcited by 72
  8. 2024A Comprehensive Survey of Foundation Models in Medicinecited by 56
  9. 2024MedImageInsight: An Open-Source Embedding Model for General Domain Medical Imagingcited by 45
  10. 2024HistGen: Histopathology Report Generation via Local-Global Feature Encoding and Cross-modal Context Interactioncited by 43
  11. 2024Automated Radiology Report Generation: A Review of Recent Advancescited by 43
  12. 2025Enhanced Contrastive Learning with Multi-view Longitudinal Data for Chest X-ray Report Generationcited by 39
38 more
  1. 2023Multimodal Image-Text Matching Improves Retrieval-based Chest X-Ray Report Generationcited by 37
  2. 2024CXPMRG-Bench: Pre-training and Benchmarking for X-ray Medical Report Generation on CheXpert Plus Datasetcited by 33
  3. 2024Dia-LLaMA: Towards Large Language Model-driven CT Report Generationcited by 32
  4. 2024HERGen: Elevating Radiology Report Generation with Longitudinal Datacited by 32
  5. 2024Towards a Holistic Framework for Multimodal Large Language Models in Three-dimensional Brain CT Report Generationcited by 30
  6. 2024Large Language Model with Region-guided Referring and Grounding for CT Report Generationcited by 27
  7. 2023Longitudinal Data and a Semantic Similarity Reward for Chest X-Ray Report Generationcited by 24
  8. 2024ReXrank: A Public Leaderboard for AI-Powered Radiology Report Generationcited by 24
  9. 2024An X-Ray Is Worth 15 Features: Sparse Autoencoders for Interpretable Radiology Report Generationcited by 23
  10. 2024ChEX: Interactive Localization and Region Description in Chest X-rayscited by 18
  11. 2024FineRadScore: A Radiology Report Line-by-Line Evaluation Technique Generating Corrections with Severity Scorescited by 18
  12. 2024A Comprehensive Survey on Evaluating Large Language Model Applications in the Medical Industrycited by 17
  13. 2024M4CXR: Exploring Multi-task Potentials of Multi-modal Large Language Models for Chest X-ray Interpretationcited by 16
  14. 2023A Systematic Review of Deep Learning-based Research on Radiology Report Generationcited by 14
  15. 2024Structural Entities Extraction and Patient Indications Incorporation for Chest X-ray Report Generationcited by 14
  16. 2023Consensus, dissensus and synergy between clinicians and specialist foundation models in radiology report generationcited by 12
  17. 2024Memory-based Cross-modal Semantic Alignment Network for Radiology Report Generationcited by 11
  18. 2024R2GenCSR: Mining Contextual and Residual Information for LLMs-based Radiology Report Generationcited by 11
  19. 2025Activating Associative Disease-Aware Vision Token Memory for LLM-Based X-ray Report Generationcited by 11
  20. 2025CoCa-CXR: Contrastive Captioners Learn Strong Temporal Structures for Chest X-Ray Vision-Language Understandingcited by 11
  21. 2024CheX-GPT: Harnessing Large Language Models for Enhanced Chest X-ray Report Labelingcited by 10
  22. 2024Factual Serialization Enhancement: A Key Innovation for Chest X-ray Report Generationcited by 10
  23. 2024e-Health CSIRO at RRG24: Entropy-Augmented Self-Critical Sequence Training for Radiology Report Generationcited by 10
  24. 2024Large Language Models in Biomedical and Health Informatics: A Review with Bibliometric Analysiscited by 9
  25. 2024HC-LLM: Historical-Constrained Large Language Models for Radiology Report Generationcited by 9
  26. 2025PriorRG: Prior-Guided Contrastive Pre-training and Coarse-to-Fine Decoding for Chest X-ray Report Generationcited by 9
  27. 2024The Impact of Auxiliary Patient Data on Automated Chest X-Ray Report Generation and How to Incorporate Itcited by 8
  28. 2025Lunguage: A Benchmark for Structured and Sequential Chest X-ray Interpretationcited by 8
  29. 2024Uncovering Knowledge Gaps in Radiology Report Generation Models through Knowledge Graphscited by 7
  30. 2025DDaTR: Dynamic Difference-aware Temporal Residual Network for Longitudinal Radiology Report Generationcited by 7
  31. 2025CXRAgent: Director-Orchestrated Multi-Stage Reasoning for Chest X-Ray Interpretationcited by 7
  32. 2026Scaling medical imaging report generation with multimodal reinforcement learningcited by 7
  33. 2024Prompt-Guided Generation of Structured Chest X-Ray Report Using a Pre-trained LLMcited by 6
  34. 2024KARGEN: Knowledge-enhanced Automated Radiology Report Generation Using Large Language Modelscited by 6
  35. 2025A Chain of Diagnosis Framework for Accurate and Explainable Radiology Report Generationcited by 6
  36. 2024EVOKE: Elevating Chest X-ray Report Generation via Multi-View Contrastive Learning and Patient-Specific Knowledgecited by 4
  37. 2024FG-CXR: A Radiologist-Aligned Gaze Dataset for Enhancing Interpretability in Chest X-Ray Report Generationcited by 4
  38. 2024Gla-AI4BioMed at RRG24: Visual Instruction-tuned Adaptation for Radiology Report Generationcited by 4

50 most cited of 93 All 93 (JSON)