mlsanityData to 7 Oct 2026

KU-DMIS-MSRA at RadSum23: Pre-trained Vision-Language Model for Radiology Report Summarization

arXiv 2307.07409 ↗First posted 10 Jul 2023cs.CL cs.AI eess.IVcited by 1

On the map: Multimodal LLMs / vision-language models → Medical vision-language models

Abstract (arXiv)

In this paper, we introduce CheXOFA, a new pre-trained vision-language model (VLM) for the chest X-ray domain. Our model is initially pre-trained on various multimodal datasets within the general domain before being transferred to the chest X-ray domain. Following a prominent VLM, we unify various domain-specific tasks into a simple sequence-to-sequence schema. It enables the model to effectively learn the required knowledge and skills from limited resources in the domain. Demonstrating superior performance on the benchmark datasets provided by the BioNLP shared task, our model benefits from its training across multiple tasks and domains. With subtle techniques including ensemble and factual calibration, our system achieves first place on the RadSum23 leaderboard for the hidden test set.

Key facts. Cited by 1 paper in the map (data through 7 Oct 2026). Placed in Medical vision-language models, one of 16 directions in Multimodal LLMs / vision-language models. Builds on 26 of its 35 references that are in the map. First posted on arXiv 10 Jul 2023.

Builds on

26 of 35 references.

Closest ancestors · 1

Same direction, newest first

  1. 2021Multi-modal Understanding and Generation for Medical Images and Text via Vision-Language Pre-Training24 May

Other references · 25

  1. 2017Attention Is All You Needcited by 59,932
  2. 2015Deep Residual Learning for Image Recognitioncited by 51,809
  3. 2019BART: Denoising Sequence-to-Sequence Pre-training for Natural Language Generation, Translation, and Comprehensioncited by 5,874
  4. 2019BERTScore: Evaluating Text Generation with BERTcited by 5,462
  5. 2022Flamingo: a Visual Language Model for Few-Shot Learningcited by 4,337
  6. 2015Neural Machine Translation of Rare Words with Subword Unitscited by 2,898
  7. 2019ViLBERT: Pretraining Task-Agnostic Visiolinguistic Representations for Vision-and-Language Taskscited by 2,219
  8. 2019CheXpert: A Large Chest Radiograph Dataset with Uncertainty Labels and Expert Comparisoncited by 1,410
  9. 2022OFA: Unifying Architectures, Tasks, and Modalities Through a Simple Sequence-to-Sequence Learning Frameworkcited by 742
  10. 2020Generating Radiology Reports via Memory-driven Transformercited by 363
  11. 2022Making the Most of Text Semantics to Improve Biomedical Vision--Language Processingcited by 336
  12. 2020CheXbert: Combining Automatic Labelers and Expert Annotations for Accurate Radiology Report Labeling Using BERTcited by 329
13 more
  1. 2021RadGraph: Extracting Clinical Entities and Relations from Radiology Reportscited by 310
  2. 2020When Radiology Report Generation Meets Knowledge Graphcited by 144
  3. 2020Improving Factual Completeness and Consistency of Image-to-Text Radiology Report Generationcited by 139
  4. 2022Multi-Modal Masked Autoencoders for Medical Vision-and-Language Pre-Trainingcited by 104
  5. 2019Optimizing the Factual Correctness of a Summary: A Study of Summarizing Radiology Reportscited by 96
  6. 2022Improving the Factual Correctness of Radiology Report Generation with Semantic Rewardscited by 82
  7. 2021Contrastive Attention for Automatic Chest X-ray Report Generationcited by 71
  8. 2018Learning to Summarize Radiology Findingscited by 54
  9. 2021Auto-Encoding Knowledge Graph for Unsupervised Medical Report Generationcited by 47
  10. 2022Graph Enhanced Contrastive Learning for Radiology Findings Summarizationcited by 24
  11. 2021Word Graph Guided Summarization for Radiology Findingscited by 19
  12. 2022Differentiable Multi-Agent Actor-Critic for Multi-Step Radiology Report Summarizationcited by 13
  13. 2022Improving Radiology Summarization with Radiograph and Anatomy Promptscited by 4

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2026: 1.

1 paper.

  1. 2026Less Is More? Selective Visual Attention to High-Importance Regions for Multimodal Radiology Summarizationcited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2025Evaluating Vision Language Models (VLMs) for Radiology: A Comprehensive Analysis
  2. 2025RadVLM: A Multitask Conversational Vision-Language Model for Radiology
  3. 2023Vision-Language Modelling For Radiological Imaging and Reports In The Low Data Regime
  4. 2025A Reality Check of Vision-Language Pre-training in Radiology: Have We Progressed Using Text?
  5. 2024D-Rax: Domain-specific Radiologic assistant leveraging multi-modal data and eXpert model predictions
  6. 2024Pretraining Vision-Language Model for Difference Visual Question Answering in Longitudinal Chest X-rays
  7. 2024CXR-Agent: Vision-language models for chest X-ray interpretation with uncertainty aware radiology reporting
  8. 2024Gla-AI4BioMed at RRG24: Visual Instruction-tuned Adaptation for Radiology Report Generation