A Survey for Biomedical Text Summarization: From Pre-trained to Large Language Models
Not on the map. Why?
Abstract (arXiv)
The exponential growth of biomedical texts such as biomedical literature and electronic health records (EHRs), poses a significant challenge for clinicians and researchers to access clinical information efficiently. To tackle this challenge, biomedical text summarization (BTS) has been proposed as a solution to support clinical information retrieval and management. BTS aims at generating concise summaries that distill key information from single or multiple biomedical documents. In recent years, the rapid advancement of fundamental natural language processing (NLP) techniques, from pre-trained language models (PLMs) to large language models (LLMs), has greatly facilitated the progress of BTS. This growth has led to numerous proposed summarization methods, datasets, and evaluation metrics, raising the need for a comprehensive and up-to-date survey for BTS. In this paper, we present a systematic review of recent advancements in BTS, leveraging cutting-edge NLP techniques from PLMs to LLMs, to help understand the latest progress, challenges, and future directions. We begin by introducing the foundational concepts of BTS, PLMs and LLMs, followed by an in-depth review of available datasets, recent approaches, and evaluation metrics in BTS. We finally discuss existing challenges and promising future directions in the era of LLMs. To facilitate the research community, we line up open resources including available datasets, recent approaches, codes, evaluation metrics, and the leaderboard in a public project: https://github.com/KenZLuo/Biomedical-Text-Summarization-Survey/tree/master. We believe that this survey will be a useful resource to researchers, allowing them to quickly track recent advancements and provide guidelines for future BTS research within the research community.
Key facts. Cited by 3 papers in the map (data through 14 Oct 2026). Builds on 76 of its 148 references that are in the map. First posted on arXiv 18 Apr 2023.
Builds on
76 of 148 references.
References · 76
- 2017Attention Is All You Needcited by 60,361
- 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,399
- 2020Language Models are Few-Shot Learnerscited by 32,419
- 2022Training language models to follow instructions with human feedbackcited by 15,971
- 2019Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformercited by 14,599
- 2023LLaMA: Open and Efficient Foundation Language Modelscited by 14,132
- 2019RoBERTa: A Robustly Optimized BERT Pretraining Approachcited by 13,372
- 2019BART: Denoising Sequence-to-Sequence Pre-training for Natural Language Generation, Translation, and Comprehensioncited by 5,893
- 2019BERTScore: Evaluating Text Generation with BERTcited by 5,512
- 2019DistilBERT, a distilled version of BERT: smaller, faster, cheaper and lightercited by 4,307
- 2021Finetuned Language Models Are Zero-Shot Learnerscited by 3,780
- 2013Distributed Representations of Words and Phrases and their Compositionalitycited by 3,474
38 more
- 2020Longformer: The Long-Document Transformercited by 3,413
- 2022Scaling Instruction-Finetuned Language Modelscited by 3,024
- 2014Sequence to Sequence Learning with Neural Networkscited by 2,986
- 2015Neural Machine Translation of Rare Words with Subword Unitscited by 2,918
- 2019ALBERT: A Lite BERT for Self-supervised Learning of Language Representationscited by 2,790
- 2023Sparks of Artificial General Intelligence: Early experiments with GPT-4cited by 2,592
- 2020Learning to summarize from human feedbackcited by 2,588
- 2022Emergent Abilities of Large Language Modelscited by 2,512
- 2018Deep contextualized word representationscited by 2,290
- 2018SentencePiece: A simple and language independent subword tokenizer and detokenizer for Neural Text Processingcited by 1,989
- 2019BioBERT: a pre-trained biomedical language representation model for biomedical text miningcited by 1,947
- 2022Large Language Models Encode Clinical Knowledgecited by 1,830
- 2020Big Bird: Transformers for Longer Sequencescited by 1,684
- 2020Don't Stop Pretraining: Adapt Language Models to Domains and Taskscited by 1,565
- 2021Multitask Prompted Training Enables Zero-Shot Task Generalizationcited by 1,470
- 2019SciBERT: A Pretrained Language Model for Scientific Textcited by 1,414
- 2019PEGASUS: Pre-training with Extracted Gap-sentences for Abstractive Summarizationcited by 1,102
- 2015Aligning Books and Movies: Towards Story-like Visual Explanations by Watching Movies and Reading Bookscited by 1,043
- 2020Domain-Specific Language Model Pretraining for Biomedical Natural Language Processingcited by 1,009
- 2019Publicly Available Clinical BERT Embeddingscited by 1,002
- 2020How Much Knowledge Can You Pack Into the Parameters of a Language Model?cited by 719
- 2023Capabilities of GPT-4 on Medical Challenge Problemscited by 638
- 2019Evaluating the Factual Consistency of Abstractive Text Summarizationcited by 584
- 2019Text Summarization with Pretrained Encoderscited by 556
- 2020Pre-trained Models for Natural Language Processing: A Surveycited by 505
- 2018A Discourse-Aware Attention Model for Abstractive Summarization of Long Documentscited by 478
- 2019A Survey on Explainable Artificial Intelligence (XAI): Towards Medical XAIcited by 404
- 2015A Neural Attention Model for Abstractive Sentence Summarizationcited by 396
- 2023Larger language models do in-context learning differentlycited by 394
- 2020CheXbert: Combining Automatic Labelers and Expert Annotations for Accurate Radiology Report Labeling Using BERTcited by 337
- 2019Transfer Learning in Biomedical Natural Language Processing: An Evaluation of BERT and ELMo on Ten Benchmarking Datasetscited by 299
- 2016SummaRuNNer: A Recurrent Neural Network based Sequence Model for Extractive Summarization of Documentscited by 246
- 2021LongT5: Efficient Text-To-Text Transformer for Long Sequencescited by 232
- 2020ProphetNet: Predicting Future N-gram for Sequence-to-Sequence Pre-trainingcited by 222
- 2020Extractive Summarization as Text Matchingcited by 189
- 2020CORD-19: The COVID-19 Open Research Datasetcited by 180
- 2021SciFive: a text-to-text transformer model for biomedical literaturecited by 99
- 2019Optimizing the Factual Correctness of a Summary: A Study of Summarizing Radiology Reportscited by 96
50 most cited of 76 All 76 (JSON)
Built on by
Citing papers per year: 2023: 1; 2024: 1; 2025: 1.
3 papers.
- 2024MEDVOC: Vocabulary Adaptation for Fine-tuning Pre-trained Language Models on Medical Text Summarizationcited by 8
- 2023Ascle: A Python Natural Language Processing Toolkit for Medical Text Generationcited by 3
- 2025Comparative Analysis of Abstractive Summarization Models for Clinical Radiology Reportscited by 0
- 2025Comparative Analysis of Abstractive Summarization Models for Clinical Radiology Reports19 Jun 2025
- 2024MEDVOC: Vocabulary Adaptation for Fine-tuning Pre-trained Language Models on Medical Text Summarization7 May 2024
- 2023Ascle: A Python Natural Language Processing Toolkit for Medical Text Generation28 Nov 2023