mlsanityData to 7 Oct 2026

Designing a Robust Radiology Report Generation System

arXiv 2411.01153 ↗First posted 2 Nov 2024cs.CV cs.AIcited by 1

Not on the map. Why?

Abstract (arXiv)

Recent advances in deep learning have enabled researchers to explore tasks at the intersection of computer vision and natural language processing, such as image captioning, visual question answering, visual dialogue, and visual language navigation. Taking inspiration from image captioning, the task of radiology report generation aims at automatically generating radiology reports by having a comprehensive understanding of medical images. However, automatically generating radiology reports from medical images is a challenging task due to the complexity, diversity, and nature of medical images. In this paper, we outline the design of a robust radiology report generation system by integrating different modules and highlighting best practices drawing upon lessons from our past work and also from relevant studies in the literature. We also discuss the impact of integrating different components to form a single integrated system. We believe that these best practices, when implemented, could improve automatic radiology report generation, augment radiologists in decision making, and expedite diagnostic workflow, in turn improve healthcare and save human lives.

Key facts. Cited by 1 paper in the map (data through 7 Oct 2026). Builds on 31 of its 62 references that are in the map. First posted on arXiv 2 Nov 2024.

Builds on

31 of 62 references.

References · 31

  1. 2017Attention Is All You Needcited by 59,932
  2. 2020An Image is Worth 16x16 Words: Transformers for Image Recognition at Scalecited by 29,915
  3. 2016Grad-CAM: Visual Explanations from Deep Networks via Gradient-based Localizationcited by 6,690
  4. 2016"Why Should I Trust You?": Explaining the Predictions of Any Classifiercited by 5,040
  5. 2014Learning Phrase Representations using RNN Encoder-Decoder for Statistical Machine Translationcited by 4,016
  6. 2015Learning Deep Features for Discriminative Localizationcited by 2,430
  7. 2014CIDEr: Consensus-based Image Description Evaluationcited by 2,027
  8. 2019CheXpert: A Large Chest Radiograph Dataset with Uncertainty Labels and Expert Comparisoncited by 1,410
  9. 2017ChestX-ray8: Hospital-scale Chest X-ray Database and Benchmarks on Weakly-Supervised Classification and Localization of Common Thorax Diseasescited by 969
  10. 2014Show and Tell: A Neural Image Caption Generatorcited by 877
  11. 2014CNN Features off-the-shelf: an Astounding Baseline for Recognitioncited by 316
  12. 2017On the Automatic Generation of Medical Imaging Reportscited by 215
19 more
  1. 2018Hybrid Retrieval-Generation Reinforced Agent for Medical Image Report Generationcited by 141
  2. 2020Improving Factual Completeness and Consistency of Image-to-Text Radiology Report Generationcited by 139
  3. 2023Interactive and Explainable Region-guided Radiology Report Generationcited by 139
  4. 2019Clinically Accurate Chest X-Ray Report Generationcited by 138
  5. 2022Foundations and Trends in Multimodal Machine Learning: Principles, Challenges, and Open Questionscited by 116
  6. 2023R2GenGPT: Radiology Report Generation with Frozen LLMscited by 111
  7. 2019Knowledge-driven Encode, Retrieve, Paraphrase for Medical Image Report Generationcited by 101
  8. 2019Automatic Radiology Report Generation based on Multi-view Image Fusion and Medical Concept Enrichmentcited by 60
  9. 2016Learning to Read Chest X-Rays: Recurrent Neural Cascade Model for Automated Image Annotationcited by 59
  10. 2017MDNet: A Semantically and Visually Interpretable Medical Image Diagnosis Networkcited by 56
  11. 2018Learning to Summarize Radiology Findingscited by 54
  12. 2023Multimodal Image-Text Matching Improves Retrieval-based Chest X-Ray Report Generationcited by 37
  13. 2023Retrieval Augmented Chest X-Ray Report Generation using OpenAI GPT modelscited by 27
  14. 2020Learning Visual-Semantic Embeddings for Reporting Abnormal Findings on Chest X-rayscited by 18
  15. 2020Quantifying the Value of Lateral Views in Deep Learning for Chest X-rayscited by 16
  16. 2021Trust It or Not: Confidence-Guided Automatic Radiology Report Generationcited by 9
  17. 2024SERPENT-VLM : Self-Refining Radiology Report Generation Using Vision Language Modelscited by 8
  18. 2024KARGEN: Knowledge-enhanced Automated Radiology Report Generation Using Large Language Modelscited by 6
  19. 2024Clinical Context-aware Radiology Report Generation from Medical Images using Transformerscited by 2

Built on by

Cited by year

Citing papers per year: 2025: 1.

1 paper.

  1. 2025Vision-Language Models for Automated Chest X-ray Interpretation: Leveraging ViT and GPT-2cited by 2