mlsanityData to 14 Oct 2026

Fine-tune BERT for Extractive Summarization

arXiv 1903.10318 ↗First posted 25 Mar 2019cs.CLcited by 122

Not on the map. Why?

Abstract (arXiv)

BERT, a pre-trained Transformer model, has achieved ground-breaking performance on multiple NLP tasks. In this paper, we describe BERTSUM, a simple variant of BERT, for extractive summarization. Our system is the state of the art on the CNN/Dailymail dataset, outperforming the previous best-performed system by 1.65 on ROUGE-L. The codes to reproduce our results are available at https://github.com/nlpyang/BertSum

Key facts. Cited by 122 papers in the map (data through 14 Oct 2026). First posted on arXiv 25 Mar 2019.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2021: 28; 2022: 21; 2023: 29; 2024: 26; 2025: 10; 2026: 8.

122 papers.

Sort:

By area:

  1. 2023Think before you speak: Training Language Models With Pause Tokenscited by 264
  2. 2022Language model compression with weighted low-rank factorizationcited by 195
  3. 2021DialogSum: A Real-Life Scenario Dialogue Summarization Datasetcited by 183
  4. 2024A Comprehensive Survey on Process-Oriented Automatic Text Summarization with Exploration of LLM-Based Methodscited by 102
  5. 2023Element-aware Summarization with Large Language Models: Expert-aligned Evaluation and Chain-of-Thought Methodcited by 76
  6. 2021DOC2PPT: Automatic Presentation Slides Generation from Scientific Documentscited by 70
  7. 2023An Iterative Optimizing Framework for Radiology Report Summarization with ChatGPTcited by 57
  8. 2023Sequential Integrated Gradients: a simple but effective method for explaining language modelscited by 52
  9. 2023How Ready are Pre-trained Abstractive Models and LLMs for Legal Case Judgement Summarization?cited by 43
  10. 2024Natural Language Processing for the Legal Domain: A Survey of Tasks, Datasets, Models, and Challengescited by 42
  11. 2024HtmlRAG: HTML is Better Than Plain Text for Modeling Retrieved Knowledge in RAG Systemscited by 37
  12. 2023LLMRec: Benchmarking Large Language Models on Recommendation Taskcited by 30
38 more
  1. 2021Noise Stability Regularization for Improving BERT Fine-tuningcited by 29
  2. 2024BIDER: Bridging Knowledge Inconsistency for Efficient Retrieval-Augmented LLMs via Key Supporting Evidencecited by 28
  3. 2022Heterformer: Transformer-based Deep Node Representation Learning on Heterogeneous Text-Rich Networkscited by 24
  4. 2022OASum: Large-Scale Open Domain Aspect-based Summarizationcited by 24
  5. 2024Two Stones Hit One Bird: Bilevel Positional Encoding for Better Length Extrapolationcited by 24
  6. 2024Applicability of Large Language Models and Generative Models for Legal Case Judgement Summarizationcited by 23
  7. 2021Enriching and Controlling Global Semantics for Text Summarizationcited by 20
  8. 2022GanLM: Encoder-Decoder Pre-training with an Auxiliary Discriminatorcited by 20
  9. 2021On the Inductive Bias of Masked Language Modeling: From Statistical to Syntactic Dependenciescited by 18
  10. 2022An Extractive-and-Abstractive Framework for Source Code Summarizationcited by 18
  11. 2023SpikeBERT: A Language Spikformer Learned from BERT with Knowledge Distillationcited by 17
  12. 2021Attention-based Clinical Note Summarizationcited by 16
  13. 2021How well do you know your summarization datasets?cited by 16
  14. 2023CodeApex: A Bilingual Programming Evaluation Benchmark for Large Language Modelscited by 15
  15. 2022Salience Allocation as Guidance for Abstractive Summarizationcited by 14
  16. 2022Numerical Optimizations for Weighted Low-rank Estimation on Language Modelcited by 14
  17. 2023Examining Bias in Opinion Summarisation Through the Perspective of Opinion Diversitycited by 14
  18. 2021Non-autoregressive Transformer-based End-to-end ASR using BERTcited by 13
  19. 2024Exploring the Efficacy of Large Language Models in Summarizing Mental Health Counseling Sessions: A Benchmark Studycited by 13
  20. 2022Improving Pre-trained Language Model Fine-tuning with Noise Stability Regularizationcited by 12
  21. 2024Few-shot clinical entity recognition in English, French and Spanish: masked language models outperform generative model promptingcited by 12
  22. 2021Speech Recognition by Simply Fine-tuning BERTcited by 11
  23. 2021UniLog: Deploy One Model and Specialize it for All Log Analysis Taskscited by 11
  24. 2021Robust Deep Reinforcement Learning for Extractive Legal Summarizationcited by 10
  25. 2024Rethinking Legal Judgement Prediction in a Realistic Scenario in the Era of Large Language Modelscited by 10
  26. 2021Neural Content Extraction for Poster Generation of Scientific Paperscited by 9
  27. 2021Model Extraction and Defenses on Generative Adversarial Networkscited by 8
  28. 2021s2s-ft: Fine-Tuning Pretrained Transformer Encoders for Sequence-to-Sequence Learningcited by 7
  29. 2022Mask More and Mask Later: Efficient Pre-training of Masked Language Models by Disentangling the [MASK] Tokencited by 7
  30. 2023Fusing Multimodal Signals on Hyper-complex Space for Extreme Abstractive Text Summarization (TL;DR) of Scientific Contentscited by 7
  31. 2023LLM Based Multi-Document Summarization Exploiting Main-Event Biased Monotone Submodular Content Extractioncited by 7
  32. 2022WikiDes: A Wikipedia-Based Dataset for Generating Short Descriptions from Paragraphscited by 6
  33. 2023Is Summary Useful or Not? An Extrinsic Human Evaluation of Text Summaries on Downstream Taskscited by 6
  34. 2024HDT: Hierarchical Document Transformercited by 6
  35. 2024SlideSpawn: An Automatic Slides Generation System for Research Publicationscited by 6
  36. 2026EMBER: Efficient Memory via Budgeted Evidence Retention for Long-Horizon Agentscited by 6
  37. 2023SciLit: A Platform for Joint Scientific Literature Discovery, Summarization and Citation Generationcited by 5
  38. 2024Semi-Supervised Dialogue Abstractive Summarization via High-Quality Pseudolabel Selectioncited by 5

50 most cited of 122 All 122 (JSON)