mlsanityData to 7 Oct 2026

Summarization is (Almost) Dead

arXiv 2309.09558 ↗First posted 18 Sep 2023cs.CLcited by 43

Not on the map. Why?

Abstract (arXiv)

How well can large language models (LLMs) generate summaries? We develop new datasets and conduct human evaluation experiments to evaluate the zero-shot generation capability of LLMs across five distinct summarization tasks. Our findings indicate a clear preference among human evaluators for LLM-generated summaries over human-written summaries and summaries generated by fine-tuned models. Specifically, LLM-generated summaries exhibit better factual consistency and fewer instances of extrinsic hallucinations. Due to the satisfactory performance of LLMs in summarization tasks (even surpassing the benchmark of reference summaries), we believe that most conventional works in the field of text summarization are no longer necessary in the era of LLMs. However, we recognize that there are still some directions worth exploring, such as the creation of novel datasets with higher quality and more reliable evaluation methods.

Key facts. Cited by 43 papers in the map (data through 7 Oct 2026). Builds on 29 of its 36 references that are in the map. First posted on arXiv 18 Sep 2023.

Builds on

29 of 36 references.

References · 29

  1. 2020Language Models are Few-Shot Learnerscited by 32,199
  2. 2022Training language models to follow instructions with human feedbackcited by 15,712
  3. 2019Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformercited by 14,491
  4. 2023LLaMA: Open and Efficient Foundation Language Modelscited by 13,958
  5. 2019BART: Denoising Sequence-to-Sequence Pre-training for Natural Language Generation, Translation, and Comprehensioncited by 5,874
  6. 2022OPT: Open Pre-trained Transformer Language Modelscited by 3,464
  7. 2022Scaling Instruction-Finetuned Language Modelscited by 2,997
  8. 2016Pointer Sentinel Mixture Modelscited by 2,992
  9. 2023G-Eval: NLG Evaluation using GPT-4 with Better Human Alignmentcited by 2,205
  10. 2020mT5: A massively multilingual pre-trained text-to-text transformercited by 1,813
  11. 2017Get To The Point: Summarization with Pointer-Generator Networkscited by 1,281
  12. 2019PEGASUS: Pre-training with Extracted Gap-sentences for Abstractive Summarizationcited by 1,100
17 more
  1. 2015Teaching Machines to Read and Comprehendcited by 1,039
  2. 2020On Faithfulness and Factuality in Abstractive Summarizationcited by 895
  3. 2021CodeT5: Identifier-aware Unified Pre-trained Encoder-Decoder Models for Code Understanding and Generationcited by 890
  4. 2023Benchmarking Large Language Models for News Summarizationcited by 444
  5. 2019Multi-News: a Large-Scale Multi-Document Summarization Dataset and Abstractive Hierarchical Modelcited by 431
  6. 2023Is ChatGPT a Good NLG Evaluator? A Preliminary Studycited by 420
  7. 2023Large Language Models Are State-of-the-Art Evaluators of Translation Qualitycited by 416
  8. 2022News Summarization and Evaluation in the Era of GPT-3cited by 324
  9. 2023Human-like Summarization Evaluation with ChatGPTcited by 126
  10. 2023Exploring the Limits of ChatGPT for Query or Aspect-based Text Summarizationcited by 122
  11. 2021MediaSum: A Large-scale Media Interview Dataset for Dialogue Summarizationcited by 104
  12. 2023Element-aware Summarization with Large Language Models: Expert-aligned Evaluation and Chain-of-Thought Methodcited by 76
  13. 2019NCLS: Neural Cross-Lingual Summarizationcited by 64
  14. 2022ClidSum: A Benchmark Dataset for Cross-Lingual Dialogue Summarizationcited by 25
  15. 2021PyTorrent: A Python Library Corpus for Large-scale Language Modelscited by 8
  16. 2023Is Summary Useful or Not? An Extrinsic Human Evaluation of Text Summaries on Downstream Taskscited by 6
  17. 2022Unsupervised Multi-Granularity Summarizationcited by 5

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2023: 6; 2024: 15; 2025: 20; 2026: 2.

43 papers.

Sort:

By area:

  1. 2023Survey on Factuality in Large Language Models: Knowledge, Retrieval and Domain-Specificitycited by 204
  2. 2023BooookScore: A systematic exploration of book-length summarization in the era of LLMscited by 137
  3. 2024A Comprehensive Survey on Process-Oriented Automatic Text Summarization with Exploration of LLM-Based Methodscited by 101
  4. 2025Learn-by-interact: A Data-Centric Framework for Self-Adaptive Agents in Realistic Environmentscited by 93
  5. 2025Large language models for automated scholarly paper review: A surveycited by 59
  6. 2025KramaBench: A Benchmark for AI Systems on Data-to-Insight Pipelines over Data Lakescited by 30
  7. 2023Learning to Break: Knowledge-Enhanced Reasoning in Multi-Agent Debate Systemcited by 26
  8. 2024Reading Subtext: Evaluating Large Language Models on Short Story Summarization with Writerscited by 19
  9. 2023User Modeling in the Era of Large Language Models: Current Research and Future Directionscited by 18
  10. 2024Preemptive Answer "Attacks" on Chain-of-Thought Reasoningcited by 16
  11. 2023Adapting Large Language Models for Education: Foundational Capabilities, Potentials, and Challengescited by 13
  12. 2024Dr-LLaVA: Visual Instruction Tuning with Symbolic Clinical Groundingcited by 10
31 more
  1. 2023Translating Legalese: Enhancing Public Understanding of Court Opinions with Legal Summarizerscited by 8
  2. 2024Key-Element-Informed sLLM Tuning for Document Summarizationcited by 8
  3. 2025Reasoning with Graphs: Structuring Implicit Knowledge to Enhance LLMs Reasoningcited by 8
  4. 2024Automated Review Generation Method Based on Large Language Modelscited by 7
  5. 2024Learning to Localize Actions in Instructional Videos with LLM-Based Multi-Pathway Text-Video Alignmentcited by 6
  6. 2024Opportunities and Challenges of Generative-AI in Financecited by 6
  7. 2024MageBench: Bridging Large Multimodal Models to Agentscited by 6
  8. 2024Characterizing Multimodal Long-form Summarization: A Case Study on Financial Reportscited by 5
  9. 2024Online Safety Analysis for LLMs: a Benchmark, an Assessment, and a Path Forwardcited by 5
  10. 2025A Cost-Effective Near-Storage Processing Solution for Offline Inference of Long-Context LLMscited by 4
  11. 2025From Idea to Implementation: Evaluating the Influence of Large Language Models in Software Development -- An Opinion Papercited by 3
  12. 2025Modeling Motivated Reasoning in Law: Evaluating Strategic Role Conditioning in LLM Summarizationcited by 3
  13. 2025Balancing Rewards in Text Summarization: Multi-Objective Reinforcement Learning via HyperVolume Optimizationcited by 3
  14. 2025Rethinking Hate Speech Detection on Social Media: Can LLMs Replace Traditional Models?cited by 2
  15. 2025Revisiting Disaggregated Large Language Model Serving for Performance and Energy Implicationscited by 2
  16. 2024AI-Enhanced Cognitive Behavioral Therapy: Deep Learning and Large Language Models for Extracting Cognitive Pathways from Social Media Textscited by 1
  17. 2025News is More than a Collection of Facts: Moral Frame Preserving News Summarizationcited by 1
  18. 2025Summarization for Generative Relation Extraction in the Microbiome Domaincited by 1
  19. 2025ClimateSOM: A Visual Analysis Workflow for Climate Ensemble Datasetscited by 1
  20. 2025Learning to Summarize by Learning to Quiz: Adversarial Agentic Collaboration for Long Document Summarizationcited by 1
  21. 2025asLLR: LLM based Leads Ranking in Auto Salescited by 1
  22. 2026From Static Benchmarks to Dynamic Protocol: Agent-Centric Text Anomaly Detection for Evaluating LLM Reasoningcited by 1
  23. 2024A Bi-consolidating Model for Joint Relational Triple Extractioncited by 0
  24. 2024PLANTS: A Novel Problem and Dataset for Summarization of Planning-Like (PL) Taskscited by 0
  25. 2024MIMDE: Exploring the Use of Synthetic vs Human Data for Evaluating Multi-Insight Multi-Document Extraction Taskscited by 0
  26. 2025MIH-TCCT: Mitigating Inconsistent Hallucinations in LLMs via Event-Driven Text-Code Cyclic Trainingcited by 0
  27. 2025Improving Multi-turn Task Completion in Task-Oriented Dialog Systems via Prompt Chaining and Fine-Grained Feedbackcited by 0
  28. 2025Grandes modelos de lenguaje: de la predicci'on de palabras a la comprensi'on?cited by 0
  29. 2025Issue Retrieval and Verification Enhanced Supplementary Code Comment Generationcited by 0
  30. 2025VA-Blueprint: Uncovering Building Blocks for Visual Analytics System Designcited by 0
  31. 2026LLMs Should Incorporate Explicit Mechanisms for Human Empathycited by 0