mlsanityData to 7 Oct 2026

Entity-level Factual Consistency of Abstractive Text Summarization

arXiv 2102.09130 ↗First posted 18 Feb 2021cs.CL cs.AIcited by 118

Not on the map. Why?

Abstract (arXiv)

A key challenge for abstractive summarization is ensuring factual consistency of the generated summary with respect to the original document. For example, state-of-the-art models trained on existing datasets exhibit entity hallucination, generating names of entities that are not present in the source document. We propose a set of new metrics to quantify the entity-level factual consistency of generated summaries and we show that the entity hallucination problem can be alleviated by simply filtering the training data. In addition, we propose a summary-worthy entity classification task to the training process as well as a joint entity and summary generation approach, which yield further improvements in entity level metrics.

Key facts. Cited by 118 papers in the map (data through 7 Oct 2026). Builds on 15 of its 20 references that are in the map. First posted on arXiv 18 Feb 2021.

Builds on

15 of 20 references.

References · 15

  1. 2017Attention Is All You Needcited by 59,932
  2. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,204
  3. 2019BART: Denoising Sequence-to-Sequence Pre-training for Natural Language Generation, Translation, and Comprehensioncited by 5,874
  4. 2019HuggingFace's Transformers: State-of-the-art Natural Language Processingcited by 2,201
  5. 2019fairseq: A Fast, Extensible Toolkit for Sequence Modelingcited by 1,486
  6. 2018Don't Give Me the Details, Just the Summary! Topic-Aware Convolutional Neural Networks for Extreme Summarizationcited by 1,308
  7. 2019PEGASUS: Pre-training with Extracted Gap-sentences for Abstractive Summarizationcited by 1,100
  8. 2016Abstractive Text Summarization Using Sequence-to-Sequence RNNs and Beyondcited by 809
  9. 2019Evaluating the Factual Consistency of Abstractive Text Summarizationcited by 578
  10. 2018Newsroom: A Dataset of 1.3 Million Summaries with Diverse Extractive Strategiescited by 292
  11. 2019Neural Text Summarization: A Critical Evaluationcited by 189
  12. 2017Faithful to the Original: Fact Aware Neural Abstractive Summarizationcited by 141
3 more
  1. 2018Dialogue Natural Language Inferencecited by 135
  2. 2017Controllable Abstractive Summarizationcited by 110
  3. 2019Don't Say That! Making Inconsistent Dialogue Unlikely with Unlikelihood Trainingcited by 110

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2021: 9; 2022: 24; 2023: 33; 2024: 31; 2025: 13; 2026: 8.

118 papers.

Sort:

By area:

  1. 2022Survey of Hallucination in Natural Language Generationcited by 2,056
  2. 2023A Survey on Hallucination in Large Language Models: Principles, Taxonomy, Challenges, and Open Questionscited by 1,582
  3. 2022Factuality Enhanced Language Models for Open-Ended Text Generationcited by 200
  4. 2021Hallucinated but Factual! Inspecting the Factuality of Hallucinations in Abstractive Summarizationcited by 128
  5. 2021CLIFF: Contrastive Learning for Improving Faithfulness and Factuality in Abstractive Summarizationcited by 120
  6. 2023Aligning Language Models with Preferences through f-divergence Minimizationcited by 118
  7. 2022Understanding Factual Errors in Summarization: Errors, Summarizers, Datasets, Error Detectorscited by 103
  8. 2023Enhancing Uncertainty-Based Hallucination Detection with Stronger Focuscited by 98
  9. 2022Evaluating Human-Language Model Interactioncited by 90
  10. 2023ChatGPT as a Factual Inconsistency Evaluator for Text Summarizationcited by 75
  11. 2024Safeguarding Large Language Models: A Surveycited by 73
  12. 2024Risk Taxonomy, Mitigation, and Assessment Benchmarks of Large Language Model Systemscited by 63
38 more
  1. 2021Improving Factual Consistency of Abstractive Summarization via Question Answeringcited by 55
  2. 2023From Sparse to Dense: GPT-4 Summarization with Chain of Density Promptingcited by 52
  3. 2024A Systematic Survey of Text Summarization: From Statistical Methods to Large Language Modelscited by 52
  4. 2022SQuALITY: Building a Long-Document Summarization Dataset the Hard Waycited by 50
  5. 2022FactPEGASUS: Factuality-Aware Pre-training and Fine-tuning for Abstractive Summarizationcited by 49
  6. 2024Merge, Ensemble, and Cooperate! A Survey on Collaborative Strategies in the Era of Large Language Modelscited by 43
  7. 2022mFACE: Multilingual Summarization with Factual Consistency Evaluationcited by 42
  8. 2023Language Models Hallucinate, but May Excel at Fact Verificationcited by 35
  9. 2023Chain of Natural Language Inference for Reducing Large Language Model Ungrounded Hallucinationscited by 34
  10. 2021Controlling Conditional Language Models without Catastrophic Forgettingcited by 32
  11. 2023Faithfulness-Aware Decoding Strategies for Abstractive Summarizationcited by 31
  12. 2023Teaching Language Models to Hallucinate Less with Synthetic Taskscited by 31
  13. 2022Learning to Revise References for Faithful Summarizationcited by 28
  14. 2022Faithfulness in Natural Language Generation: A Systematic Survey of Analysis, Evaluation and Optimization Methodscited by 27
  15. 2022Don't Say What You Don't Know: Improving the Consistency of Abstractive Summarization by Constraining Beam Searchcited by 23
  16. 2022Improving the Faithfulness of Abstractive Summarization via Entity Coverage Controlcited by 21
  17. 2022Faithful to the Document or to the World? Mitigating Hallucinations via Entity-linked Knowledge in Abstractive Summarizationcited by 20
  18. 2025VeriSafe Agent: Safeguarding Mobile GUI Agent via Logic-based Action Verificationcited by 20
  19. 2024Comparing Hallucination Detection Metrics for Multilingual Generationcited by 18
  20. 2024An Audit on the Perspectives and Challenges of Hallucinations in NLPcited by 17
  21. 2025GraphEval: A Lightweight Graph-Based LLM Framework for Idea Evaluationcited by 17
  22. 2024Multi-News+: Cost-efficient Dataset Cleansing via LLM-based Data Annotationcited by 16
  23. 2022Shortcomings of Question Answering Based Factuality Frameworks for Error Localizationcited by 15
  24. 2022Improving Factual Consistency in Summarization with Compression-Based Post-Editingcited by 13
  25. 2023Reliability Check: An Analysis of GPT-3's Response to Sensitive Topics and Prompt Wordingcited by 13
  26. 2022Contrastive Error Attribution for Finetuned Language Modelscited by 12
  27. 2023Evaluating Large Language Models for Health-related Queries with Presuppositionscited by 12
  28. 2021CaPE: Contrastive Parameter Ensembling for Reducing Hallucination in Abstractive Summarizationcited by 11
  29. 2022EA2E: Improving Consistency with Event Awareness for Document-Level Argument Extractioncited by 11
  30. 2022Entity-based SpanCopy for Abstractive Summarization to Improve the Factual Consistencycited by 11
  31. 2023Do Multi-Document Summarization Models Synthesize?cited by 11
  32. 2022DeepGen: Diverse Search Ad Generation and Real-Time Customizationcited by 10
  33. 2023KNN-LM Does Not Improve Open-ended Text Generationcited by 10
  34. 2022Questioning the Validity of Summarization Datasets and Improving Their Factual Consistencycited by 9
  35. 2023Generating EDU Extracts for Plan-Guided Summary Re-Rankingcited by 9
  36. 2024A review of faithfulness metrics for hallucination assessment in Large Language Modelscited by 9
  37. 2023What are the Desired Characteristics of Calibration Sets? Identifying Correlates on Long Form Scientific Summarizationcited by 8
  38. 2024A Survey on Responsible Generative AI: What to Generate and What Notcited by 8

50 most cited of 118 All 118 (JSON)