mlsanityData to 14 Oct 2026

What Have We Achieved on Text Summarization?

arXiv 2010.04529 ↗First posted 9 Oct 2020cs.CLcited by 65

Not on the map. Why?

Abstract (arXiv)

Deep learning has led to significant improvement in text summarization with various methods investigated and improved ROUGE scores reported over the years. However, gaps still exist between summaries produced by automatic summarizers and human professionals. Aiming to gain more understanding of summarization systems with respect to their strengths and limits on a fine-grained syntactic and semantic level, we consult the Multidimensional Quality Metric(MQM) and quantify 8 major sources of errors on 10 representative summarization models manually. Primarily, we find that 1) under similar settings, extractive summarizers are in general better than their abstractive counterparts thanks to strength in faithfulness and factual-consistency; 2) milestone techniques such as copy, coverage and hybrid extractive/abstractive methods do bring specific improvements but also demonstrate limitations; 3) pre-training techniques, and in particular sequence-to-sequence pre-training, are highly effective for improving text summarization, with BART giving the best results.

Key facts. Cited by 65 papers in the map (data through 14 Oct 2026). First posted on arXiv 9 Oct 2020.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2021: 7; 2022: 18; 2023: 18; 2024: 17; 2025: 3; 2026: 2.

65 papers.

Sort:

By area:

  1. 2021SummaC: Re-Visiting NLI-based Models for Inconsistency Detection in Summarizationcited by 407
  2. 2023AlignScore: Evaluating Factual Consistency with a Unified Alignment Functioncited by 288
  3. 2024MiniCheck: Efficient Fact-Checking of LLMs on Grounding Documentscited by 257
  4. 2021QAFactEval: Improved QA-Based Factual Consistency Evaluation for Summarizationcited by 236
  5. 2022Revisiting the Gold Standard: Grounding Summarization Evaluation with Robust Human Evaluationcited by 142
  6. 2022Understanding Factual Errors in Summarization: Errors, Summarizers, Datasets, Error Detectorscited by 104
  7. 2023Enhancing Uncertainty-Based Hallucination Detection with Stronger Focuscited by 99
  8. 2023ChatGPT as a Factual Inconsistency Evaluator for Text Summarizationcited by 75
  9. 2022An Empirical Survey on Long Document Summarization: Datasets, Models and Metricscited by 74
  10. 2023Benchmarking Generation and Evaluation Capabilities of Large Language Models for Instruction Controllable Summarizationcited by 69
  11. 2022ECTSum: A New Benchmark Dataset For Bullet Point Summarization of Long Earnings Call Transcriptscited by 59
  12. 2021Text-to-Table: A New Way of Information Extractioncited by 47
38 more
  1. 2021Keep it Simple: Unsupervised Simplification of Multi-Paragraph Textcited by 45
  2. 2022Stretching Sentence-pair NLI Models to Reason over Long Documents and Clusterscited by 44
  3. 2022Submodularity In Machine Learning and Artificial Intelligencecited by 38
  4. 2022On Improving Summarization Factual Consistency from Natural Language Feedbackcited by 38
  5. 2025Rating Roulette: Self-Inconsistency in LLM-As-A-Judge Frameworkscited by 38
  6. 2024Analyzing LLM Behavior in Dialogue Summarization: Unveiling Circumstantial Hallucination Trendscited by 25
  7. 2022Legal Case Document Summarization: Extractive and Abstractive Methods and their Evaluationcited by 24
  8. 2022Don't Say What You Don't Know: Improving the Consistency of Abstractive Summarization by Constraining Beam Searchcited by 23
  9. 2024Fine-Grained Natural Language Inference Based Faithfulness Evaluation for Diverse Summarisation Taskscited by 23
  10. 2023Zero-shot Faithfulness Evaluation for Text Summarization with Foundation Language Modelcited by 21
  11. 2025Benchmarking LLM Faithfulness in RAG with Evolving Leaderboardscited by 20
  12. 2024Themis: A Reference-free NLG Evaluation Language Model with Flexibility and Interpretabilitycited by 19
  13. 2023Towards Better Evaluation of Instruction-Following: A Case-Study in Summarizationcited by 18
  14. 2023LongDocFACTScore: Evaluating the Factuality of Long Document Abstractive Summarisationcited by 16
  15. 2023Investigating Hallucinations in Pruned Large Language Models for Abstractive Summarizationcited by 16
  16. 2024METAL: Towards Multilingual Meta-Evaluationcited by 15
  17. 2021Generating Informative Conclusions for Argumentative Textscited by 14
  18. 2022Revisiting text decomposition methods for NLI-based factuality scoring of summariescited by 14
  19. 2023A Meta-Evaluation of Faithfulness Metrics for Long-Form Hospital-Course Summarizationcited by 14
  20. 2022Improving Factual Consistency in Summarization with Compression-Based Post-Editingcited by 13
  21. 2022Analyzing and Evaluating Faithfulness in Dialogue Summarizationcited by 12
  22. 2025Evaluating Small Language Models for News Summarization: Implications and Factors Influencing Performancecited by 12
  23. 2023AMRFact: Enhancing Summarization Factuality Evaluation with AMR-Driven Negative Samples Generationcited by 11
  24. 2024Do Automatic Factuality Metrics Measure Factuality? A Critical Evaluationcited by 11
  25. 2023Evaluating Factual Consistency of Summaries with Large Language Modelscited by 10
  26. 2023Annotating and Detecting Fine-grained Factual Errors for Dialogue Summarizationcited by 10
  27. 2024Evaluating the Factuality of Zero-shot Summarizers Across Varied Domainscited by 10
  28. 2022Questioning the Validity of Summarization Datasets and Improving Their Factual Consistencycited by 9
  29. 2023Can LMs Generalize to Future Data? An Empirical Analysis on Text Summarizationcited by 9
  30. 2024STORYSUMM: Evaluating Faithfulness in Story Summarizationcited by 9
  31. 2024A Survey of Automatic Hallucination Evaluation on Natural Language Generationcited by 7
  32. 2022NewsPod: Automatic and Interactive News Podcastscited by 6
  33. 2021Estimation of Summary-to-Text Inconsistency by Mismatched Embeddingscited by 5
  34. 2021TopicRefine: Joint Topic Prediction and Dialogue Response Generation for Multi-turn End-to-End Dialogue Systemcited by 5
  35. 2022Methodological reflections for AI alignment research using human feedbackcited by 5
  36. 2024Identifying Factual Inconsistencies in Summaries: Grounding LLM Inference via Task Taxonomycited by 5
  37. 2024Faithful Chart Summarization with ChaTS-Picited by 5
  38. 2024Using Similarity to Evaluate Factual Consistency in Summariescited by 5

50 most cited of 65 All 65 (JSON)