mlsanityData to 14 Oct 2026

A Survey for Biomedical Text Summarization: From Pre-trained to Large Language Models

arXiv 2304.08763 ↗First posted 18 Apr 2023cs.CLcited by 3

Not on the map. Why?

Abstract (arXiv)

The exponential growth of biomedical texts such as biomedical literature and electronic health records (EHRs), poses a significant challenge for clinicians and researchers to access clinical information efficiently. To tackle this challenge, biomedical text summarization (BTS) has been proposed as a solution to support clinical information retrieval and management. BTS aims at generating concise summaries that distill key information from single or multiple biomedical documents. In recent years, the rapid advancement of fundamental natural language processing (NLP) techniques, from pre-trained language models (PLMs) to large language models (LLMs), has greatly facilitated the progress of BTS. This growth has led to numerous proposed summarization methods, datasets, and evaluation metrics, raising the need for a comprehensive and up-to-date survey for BTS. In this paper, we present a systematic review of recent advancements in BTS, leveraging cutting-edge NLP techniques from PLMs to LLMs, to help understand the latest progress, challenges, and future directions. We begin by introducing the foundational concepts of BTS, PLMs and LLMs, followed by an in-depth review of available datasets, recent approaches, and evaluation metrics in BTS. We finally discuss existing challenges and promising future directions in the era of LLMs. To facilitate the research community, we line up open resources including available datasets, recent approaches, codes, evaluation metrics, and the leaderboard in a public project: https://github.com/KenZLuo/Biomedical-Text-Summarization-Survey/tree/master. We believe that this survey will be a useful resource to researchers, allowing them to quickly track recent advancements and provide guidelines for future BTS research within the research community.

Key facts. Cited by 3 papers in the map (data through 14 Oct 2026). Builds on 76 of its 148 references that are in the map. First posted on arXiv 18 Apr 2023.

Builds on

76 of 148 references.

References · 76

  1. 2017Attention Is All You Needcited by 60,361
  2. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,399
  3. 2020Language Models are Few-Shot Learnerscited by 32,419
  4. 2022Training language models to follow instructions with human feedbackcited by 15,971
  5. 2019Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformercited by 14,599
  6. 2023LLaMA: Open and Efficient Foundation Language Modelscited by 14,132
  7. 2019RoBERTa: A Robustly Optimized BERT Pretraining Approachcited by 13,372
  8. 2019BART: Denoising Sequence-to-Sequence Pre-training for Natural Language Generation, Translation, and Comprehensioncited by 5,893
  9. 2019BERTScore: Evaluating Text Generation with BERTcited by 5,512
  10. 2019DistilBERT, a distilled version of BERT: smaller, faster, cheaper and lightercited by 4,307
  11. 2021Finetuned Language Models Are Zero-Shot Learnerscited by 3,780
  12. 2013Distributed Representations of Words and Phrases and their Compositionalitycited by 3,474
38 more
  1. 2020Longformer: The Long-Document Transformercited by 3,413
  2. 2022Scaling Instruction-Finetuned Language Modelscited by 3,024
  3. 2014Sequence to Sequence Learning with Neural Networkscited by 2,986
  4. 2015Neural Machine Translation of Rare Words with Subword Unitscited by 2,918
  5. 2019ALBERT: A Lite BERT for Self-supervised Learning of Language Representationscited by 2,790
  6. 2023Sparks of Artificial General Intelligence: Early experiments with GPT-4cited by 2,592
  7. 2020Learning to summarize from human feedbackcited by 2,588
  8. 2022Emergent Abilities of Large Language Modelscited by 2,512
  9. 2018Deep contextualized word representationscited by 2,290
  10. 2018SentencePiece: A simple and language independent subword tokenizer and detokenizer for Neural Text Processingcited by 1,989
  11. 2019BioBERT: a pre-trained biomedical language representation model for biomedical text miningcited by 1,947
  12. 2022Large Language Models Encode Clinical Knowledgecited by 1,830
  13. 2020Big Bird: Transformers for Longer Sequencescited by 1,684
  14. 2020Don't Stop Pretraining: Adapt Language Models to Domains and Taskscited by 1,565
  15. 2021Multitask Prompted Training Enables Zero-Shot Task Generalizationcited by 1,470
  16. 2019SciBERT: A Pretrained Language Model for Scientific Textcited by 1,414
  17. 2019PEGASUS: Pre-training with Extracted Gap-sentences for Abstractive Summarizationcited by 1,102
  18. 2015Aligning Books and Movies: Towards Story-like Visual Explanations by Watching Movies and Reading Bookscited by 1,043
  19. 2020Domain-Specific Language Model Pretraining for Biomedical Natural Language Processingcited by 1,009
  20. 2019Publicly Available Clinical BERT Embeddingscited by 1,002
  21. 2020How Much Knowledge Can You Pack Into the Parameters of a Language Model?cited by 719
  22. 2023Capabilities of GPT-4 on Medical Challenge Problemscited by 638
  23. 2019Evaluating the Factual Consistency of Abstractive Text Summarizationcited by 584
  24. 2019Text Summarization with Pretrained Encoderscited by 556
  25. 2020Pre-trained Models for Natural Language Processing: A Surveycited by 505
  26. 2018A Discourse-Aware Attention Model for Abstractive Summarization of Long Documentscited by 478
  27. 2019A Survey on Explainable Artificial Intelligence (XAI): Towards Medical XAIcited by 404
  28. 2015A Neural Attention Model for Abstractive Sentence Summarizationcited by 396
  29. 2023Larger language models do in-context learning differentlycited by 394
  30. 2020CheXbert: Combining Automatic Labelers and Expert Annotations for Accurate Radiology Report Labeling Using BERTcited by 337
  31. 2019Transfer Learning in Biomedical Natural Language Processing: An Evaluation of BERT and ELMo on Ten Benchmarking Datasetscited by 299
  32. 2016SummaRuNNer: A Recurrent Neural Network based Sequence Model for Extractive Summarization of Documentscited by 246
  33. 2021LongT5: Efficient Text-To-Text Transformer for Long Sequencescited by 232
  34. 2020ProphetNet: Predicting Future N-gram for Sequence-to-Sequence Pre-trainingcited by 222
  35. 2020Extractive Summarization as Text Matchingcited by 189
  36. 2020CORD-19: The COVID-19 Open Research Datasetcited by 180
  37. 2021SciFive: a text-to-text transformer model for biomedical literaturecited by 99
  38. 2019Optimizing the Factual Correctness of a Summary: A Study of Summarizing Radiology Reportscited by 96

50 most cited of 76 All 76 (JSON)

Built on by

Cited by year

Citing papers per year: 2023: 1; 2024: 1; 2025: 1.

3 papers.