mlsanityData to 14 Oct 2026

shs-nlp at RadSum23: Domain-Adaptive Pre-training of Instruction-tuned LLMs for Radiology Report Impression Generation

arXiv 2306.03264 ↗First posted 5 Jun 2023cs.CLcited by 7

Not on the map. Why?

Abstract (arXiv)

Instruction-tuned generative Large language models (LLMs) like ChatGPT and Bloomz possess excellent generalization abilities, but they face limitations in understanding radiology reports, particularly in the task of generating the IMPRESSIONS section from the FINDINGS section. They tend to generate either verbose or incomplete IMPRESSIONS, mainly due to insufficient exposure to medical text data during training. We present a system which leverages large-scale medical text data for domain-adaptive pre-training of instruction-tuned LLMs to enhance its medical knowledge and performance on specific medical tasks. We show that this system performs better in a zero-shot setting than a number of pretrain-and-finetune adaptation methods on the IMPRESSIONS generation task, and ranks 1st among participating systems in Task 1B: Radiology Report Summarization at the BioNLP 2023 workshop.

Key facts. Cited by 7 papers in the map (data through 14 Oct 2026). Builds on 13 of its 31 references that are in the map. First posted on arXiv 5 Jun 2023.

Builds on

13 of 31 references.

References · 13

  1. 2019BERTScore: Evaluating Text Generation with BERTcited by 5,512
  2. 2021On the Opportunities and Risks of Foundation Modelscited by 4,033
  3. 2022BLOOM: A 176B-Parameter Open-Access Multilingual Language Modelcited by 1,984
  4. 2019BioBERT: a pre-trained biomedical language representation model for biomedical text miningcited by 1,947
  5. 2019Megatron-LM: Training Multi-Billion Parameter Language Models Using Model Parallelismcited by 1,868
  6. 2020Don't Stop Pretraining: Adapt Language Models to Domains and Taskscited by 1,565
  7. 2019CheXpert: A Large Chest Radiograph Dataset with Uncertainty Labels and Expert Comparisoncited by 1,423
  8. 2022Improving the Factual Correctness of Radiology Report Generation with Semantic Rewardscited by 83
  9. 2018Learning to Summarize Radiology Findingscited by 54
  10. 2022Differentiable Multi-Agent Actor-Critic for Multi-Step Radiology Report Summarizationcited by 13
  11. 2023FactReranker: Fact-guided Reranker for Faithful Radiology Report Summarizationcited by 6
  12. 2023RadLing: Towards Efficient Radiology Report Understandingcited by 5
1 more
  1. 2021Few-Shot Learning of an Interleaved Text Summarization Model by Pretraining with Synthetic Datacited by 4

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2023: 3; 2024: 2; 2025: 1; 2026: 1.

7 papers.

  1. 2023Large language models in medicine: the potentials and pitfallscited by 21
  2. 2023Walking a Tightrope -- Evaluating Large Language Models in High-Risk Domainscited by 15
  3. 2024Fusion of Domain-Adapted Vision and Language Models for Medical Visual Question Answeringcited by 5
  4. 2024Efficient Language Adaptive Pre-training: Extending State-of-the-Art Large Language Models for Polishcited by 4
  5. 2025Natural Language Generation in Healthcare: A Review of Methods and Applicationscited by 3
  6. 2023Generation of Radiology Findings in Chest X-Ray by Leveraging Collaborative Knowledgecited by 2
  7. 2026Less Is More? Selective Visual Attention to High-Importance Regions for Multimodal Radiology Summarizationcited by 0