mlsanityData to 7 Oct 2026

The Factual Inconsistency Problem in Abstractive Text Summarization: A Survey

arXiv 2104.14839 ↗First posted 30 Apr 2021cs.CLcited by 79

Not on the map. Why?

Abstract (arXiv)

Recently, various neural encoder-decoder models pioneered by Seq2Seq framework have been proposed to achieve the goal of generating more abstractive summaries by learning to map input text to output text. At a high level, such neural models can freely generate summaries without any constraint on the words or phrases used. Moreover, their format is closer to human-edited summaries and output is more readable and fluent. However, the neural model's abstraction ability is a double-edged sword. A commonly observed problem with the generated summaries is the distortion or fabrication of factual information in the article. This inconsistency between the original text and the summary has caused various concerns over its applicability, and the previous evaluation methods of text summarization are not suitable for this issue. In response to the above problems, the current research direction is predominantly divided into two categories, one is to design fact-aware evaluation metrics to select outputs without factual inconsistency errors, and the other is to develop new summarization systems towards factual consistency. In this survey, we focus on presenting a comprehensive review of these fact-specific evaluation methods and text summarization models.

Key facts. Cited by 79 papers in the map (data through 7 Oct 2026). Builds on 29 of its 50 references that are in the map. First posted on arXiv 30 Apr 2021.

Builds on

29 of 50 references.

References · 29

  1. 2017Attention Is All You Needcited by 59,932
  2. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,204
  3. 2019BART: Denoising Sequence-to-Sequence Pre-training for Natural Language Generation, Translation, and Comprehensioncited by 5,874
  4. 2019BERTScore: Evaluating Text Generation with BERTcited by 5,462
  5. 2018Don't Give Me the Details, Just the Summary! Topic-Aware Convolutional Neural Networks for Extreme Summarizationcited by 1,308
  6. 2017Get To The Point: Summarization with Pointer-Generator Networkscited by 1,281
  7. 2018Hierarchical Neural Story Generationcited by 1,182
  8. 2015Teaching Machines to Read and Comprehendcited by 1,039
  9. 2020On Faithfulness and Factuality in Abstractive Summarizationcited by 895
  10. 2016Abstractive Text Summarization Using Sequence-to-Sequence RNNs and Beyondcited by 809
  11. 2019Evaluating the Factual Consistency of Abstractive Text Summarizationcited by 578
  12. 2020Asking and Answering Questions to Evaluate the Factual Consistency of Summariescited by 376
17 more
  1. 2020FEQA: A Question Answering Evaluation Framework for Faithfulness Assessment in Abstractive Summarizationcited by 292
  2. 2020Detecting Hallucinated Content in Conditional Neural Sequence Generationcited by 151
  3. 2017Faithful to the Original: Fact Aware Neural Abstractive Summarizationcited by 141
  4. 2020Factual Error Correction for Abstractive Summarization Modelscited by 108
  5. 2019Assessing The Factual Accuracy of Generated Textcited by 103
  6. 2019Optimizing the Factual Correctness of a Summary: A Study of Summarizing Radiology Reportscited by 96
  7. 2016Reward Augmented Maximum Likelihood for Neural Structured Predictioncited by 80
  8. 2020GO FIGURE: A Meta Evaluation of Factuality in Summarizationcited by 75
  9. 2020Knowledge Graph-Augmented Abstractive Summarization with Semantic-Driven Cloze Rewardcited by 66
  10. 2020Multi-Fact Correction in Abstractive Text Summarizationcited by 66
  11. 2020Constrained Abstractive Summarization: Preserving Factual Consistency with Constrained Generationcited by 26
  12. 2020FFCI: A Framework for Interpretable Automatic Evaluation of Summarizationcited by 24
  13. 2020Improving Truthfulness of Headline Generationcited by 20
  14. 2020Mind The Facts: Knowledge-Boosted Coherent Abstractive Text Summarizationcited by 13
  15. 2020Enhancing Factual Consistency of Abstractive Summarizationcited by 10
  16. 2020Looking Beyond Sentence-Level Natural Language Inference for Downstream Taskscited by 6
  17. 2021Nutri-bullets: Summarizing Health Studies by Composing Segmentscited by 6

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2021: 3; 2022: 9; 2023: 29; 2024: 23; 2025: 9; 2026: 6.

79 papers.

Sort:

By area:

  1. 2022Survey of Hallucination in Natural Language Generationcited by 2,056
  2. 2023Evaluating Object Hallucination in Large Vision-Language Modelscited by 1,696
  3. 2023A Survey on Hallucination in Large Language Models: Principles, Taxonomy, Challenges, and Open Questionscited by 1,582
  4. 2023SelfCheckGPT: Zero-Resource Black-Box Hallucination Detection for Generative Large Language Modelscited by 787
  5. 2023CRITIC: Large Language Models Can Self-Correct with Tool-Interactive Critiquingcited by 751
  6. 2024Hallucination is Inevitable: An Innate Limitation of Large Language Modelscited by 329
  7. 2023A Stitch in Time Saves Nine: Detecting and Mitigating Hallucinations of LLMs by Validating Low-Confidence Generationcited by 202
  8. 2022Factuality Enhanced Language Models for Open-Ended Text Generationcited by 200
  9. 2023"Kelly is a Warm Person, Joseph is a Role Model": Gender Biases in LLM-Generated Reference Letterscited by 196
  10. 2024Multi-Modal Hallucination Control by Visual Information Groundingcited by 196
  11. 2023Self-contradictory Hallucinations of Large Language Models: Evaluation, Detection and Mitigationcited by 123
  12. 2023Generating Benchmarks for Factuality Evaluation of Language Modelscited by 103
38 more
  1. 2023Enhancing Uncertainty-Based Hallucination Detection with Stronger Focuscited by 98
  2. 2024VideoHallucer: Evaluating Intrinsic and Extrinsic Hallucinations in Large Video-Language Modelscited by 88
  3. 2023ChatGPT as a Factual Inconsistency Evaluator for Text Summarizationcited by 75
  4. 2023Mitigating Fine-Grained Hallucination by Fine-Tuning Large Vision-Language Models with Caption Rewritescited by 61
  5. 2021A Survey on Dialogue Summarization: Recent Advances and New Frontierscited by 51
  6. 2024Hallucination Detection: Robustly Discerning Reliable Answers in Large Language Modelscited by 44
  7. 2023Is Stack Overflow Obsolete? An Empirical Study of the Characteristics of ChatGPT Answers to Stack Overflow Questionscited by 42
  8. 2022Correcting Diverse Factual Errors in Abstractive Summarization via Post-Editing and Language Model Infillingcited by 38
  9. 2024AI Hallucinations: A Misnomer Worth Clarifyingcited by 34
  10. 2024Confidence Under the Hood: An Investigation into the Confidence-Probability Alignment in Large Language Modelscited by 34
  11. 2025Treble Counterfactual VLMs: A Causal Approach to Hallucinationcited by 33
  12. 2023Teaching Language Models to Hallucinate Less with Synthetic Taskscited by 31
  13. 2023MQAG: Multiple-choice Question Answering and Generation for Assessing Information Consistency in Summarizationcited by 30
  14. 2024KnowHalu: Hallucination Detection via Multi-Form Knowledge Based Factual Checkingcited by 26
  15. 2024Mitigating Large Language Model Hallucination with Faithful Finetuningcited by 24
  16. 2023Are Large Language Models Reliable Judges? A Study on the Factuality Evaluation Capabilities of LLMscited by 23
  17. 2024Small Language Model Can Self-correctcited by 23
  18. 2024CodeMirage: Hallucinations in Code Generated by Large Language Modelscited by 23
  19. 2023LongDocFACTScore: Evaluating the Factuality of Long Document Abstractive Summarisationcited by 16
  20. 2021Investigating Crowdsourcing Protocols for Evaluating the Factual Consistency of Summariescited by 15
  21. 2023Towards Unifying Multi-Lingual and Cross-Lingual Summarizationcited by 15
  22. 2025Hallucination to Truth: A Review of Fact-Checking and Factuality Evaluation in Large Language Modelscited by 13
  23. 2023Evaluating Generative Ad Hoc Information Retrievalcited by 11
  24. 2021CO2Sum:Contrastive Learning for Factual-Consistent Abstractive Summarizationcited by 9
  25. 2022Abstractive Meeting Summarization: A Surveycited by 9
  26. 2022Questioning the Validity of Summarization Datasets and Improving Their Factual Consistencycited by 9
  27. 2023FaMeSumm: Investigating and Improving Faithfulness of Medical Summarizationcited by 9
  28. 2025Bring Your Own Knowledge: A Survey of Methods for LLM Knowledge Expansioncited by 8
  29. 2024A Survey of Automatic Hallucination Evaluation on Natural Language Generationcited by 7
  30. 2023Active Learning for Abstractive Text Summarizationcited by 6
  31. 2023Tackling Hallucinations in Neural Chart Summarizationcited by 6
  32. 2023Aligning Large Language Models for Clinical Taskscited by 6
  33. 2024Strong hallucinations from negation and how to fix themcited by 6
  34. 2024A Data-Centric Approach To Generate Faithful and High Quality Patient Summaries with Large Language Modelscited by 6
  35. 2024Factual Dialogue Summarization via Learning from Large Language Modelscited by 6
  36. 2023Challenges in Domain-Specific Abstractive Summarization and How to Overcome themcited by 5
  37. 2023Sequence-Level Certainty Reduces Hallucination In Knowledge-Grounded Dialogue Generationcited by 5
  38. 2023Factored Verification: Detecting and Reducing Hallucination in Summaries of Academic Paperscited by 4

50 most cited of 79 All 79 (JSON)