# The Factual Inconsistency Problem in Abstractive Text Summarization: A Survey

arXiv: https://arxiv.org/abs/2104.14839 · HTML: https://mlsanity.com/paper/2104.14839/

## Key facts

- Cited by 79 papers in the map (data through 14 Oct 2026).
- Builds on 29 of its 50 references that are in the map.
- First posted on arXiv 30 Apr 2021.

## Builds on (most cited first, 29 of 29)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [BART: Denoising Sequence-to-Sequence Pre-training for Natural Language Generation, Translation, and Comprehension](https://mlsanity.com/paper/1910.13461/) (2019)
- [BERTScore: Evaluating Text Generation with BERT](https://mlsanity.com/paper/1904.09675/) (2019)
- [Don't Give Me the Details, Just the Summary! Topic-Aware Convolutional Neural Networks for Extreme Summarization](https://mlsanity.com/paper/1808.08745/) (2018)
- [Get To The Point: Summarization with Pointer-Generator Networks](https://mlsanity.com/paper/1704.04368/) (2017)
- [Hierarchical Neural Story Generation](https://mlsanity.com/paper/1805.04833/) (2018)
- [Teaching Machines to Read and Comprehend](https://mlsanity.com/paper/1506.03340/) (2015)
- [On Faithfulness and Factuality in Abstractive Summarization](https://mlsanity.com/paper/2005.00661/) (2020)
- [Abstractive Text Summarization Using Sequence-to-Sequence RNNs and Beyond](https://mlsanity.com/paper/1602.06023/) (2016)
- [Evaluating the Factual Consistency of Abstractive Text Summarization](https://mlsanity.com/paper/1910.12840/) (2019)
- [Asking and Answering Questions to Evaluate the Factual Consistency of Summaries](https://mlsanity.com/paper/2004.04228/) (2020)
- [FEQA: A Question Answering Evaluation Framework for Faithfulness Assessment in Abstractive Summarization](https://mlsanity.com/paper/2005.03754/) (2020)
- [Detecting Hallucinated Content in Conditional Neural Sequence Generation](https://mlsanity.com/paper/2011.02593/) (2020)
- [Faithful to the Original: Fact Aware Neural Abstractive Summarization](https://mlsanity.com/paper/1711.04434/) (2017)
- [Factual Error Correction for Abstractive Summarization Models](https://mlsanity.com/paper/2010.08712/) (2020)
- [Assessing The Factual Accuracy of Generated Text](https://mlsanity.com/paper/1905.13322/) (2019)
- [Optimizing the Factual Correctness of a Summary: A Study of Summarizing Radiology Reports](https://mlsanity.com/paper/1911.02541/) (2019)
- [Reward Augmented Maximum Likelihood for Neural Structured Prediction](https://mlsanity.com/paper/1609.00150/) (2016)
- [GO FIGURE: A Meta Evaluation of Factuality in Summarization](https://mlsanity.com/paper/2010.12834/) (2020)
- [Knowledge Graph-Augmented Abstractive Summarization with Semantic-Driven Cloze Reward](https://mlsanity.com/paper/2005.01159/) (2020)
- [Multi-Fact Correction in Abstractive Text Summarization](https://mlsanity.com/paper/2010.02443/) (2020)
- [Constrained Abstractive Summarization: Preserving Factual Consistency with Constrained Generation](https://mlsanity.com/paper/2010.12723/) (2020)
- [FFCI: A Framework for Interpretable Automatic Evaluation of Summarization](https://mlsanity.com/paper/2011.13662/) (2020)
- [Improving Truthfulness of Headline Generation](https://mlsanity.com/paper/2005.00882/) (2020)
- [Mind The Facts: Knowledge-Boosted Coherent Abstractive Text Summarization](https://mlsanity.com/paper/2006.15435/) (2020)
- [Enhancing Factual Consistency of Abstractive Summarization](https://mlsanity.com/paper/2003.08612/) (2020)
- [Looking Beyond Sentence-Level Natural Language Inference for Downstream Tasks](https://mlsanity.com/paper/2009.09099/) (2020)
- [Nutri-bullets: Summarizing Health Studies by Composing Segments](https://mlsanity.com/paper/2103.11921/) (2021)

## Built on by (most cited first, 30 of 79)

- [Survey of Hallucination in Natural Language Generation](https://mlsanity.com/paper/2202.03629/) (2022)
- [Evaluating Object Hallucination in Large Vision-Language Models](https://mlsanity.com/paper/2305.10355/) (2023)
- [A Survey on Hallucination in Large Language Models: Principles, Taxonomy, Challenges, and Open Questions](https://mlsanity.com/paper/2311.05232/) (2023)
- [SelfCheckGPT: Zero-Resource Black-Box Hallucination Detection for Generative Large Language Models](https://mlsanity.com/paper/2303.08896/) (2023)
- [CRITIC: Large Language Models Can Self-Correct with Tool-Interactive Critiquing](https://mlsanity.com/paper/2305.11738/) (2023)
- [Hallucination is Inevitable: An Innate Limitation of Large Language Models](https://mlsanity.com/paper/2401.11817/) (2024)
- [Multi-Modal Hallucination Control by Visual Information Grounding](https://mlsanity.com/paper/2403.14003/) (2024)
- [A Stitch in Time Saves Nine: Detecting and Mitigating Hallucinations of LLMs by Validating Low-Confidence Generation](https://mlsanity.com/paper/2307.03987/) (2023)
- [Factuality Enhanced Language Models for Open-Ended Text Generation](https://mlsanity.com/paper/2206.04624/) (2022)
- ["Kelly is a Warm Person, Joseph is a Role Model": Gender Biases in LLM-Generated Reference Letters](https://mlsanity.com/paper/2310.09219/) (2023)
- [Self-contradictory Hallucinations of Large Language Models: Evaluation, Detection and Mitigation](https://mlsanity.com/paper/2305.15852/) (2023)
- [Generating Benchmarks for Factuality Evaluation of Language Models](https://mlsanity.com/paper/2307.06908/) (2023)
- [Enhancing Uncertainty-Based Hallucination Detection with Stronger Focus](https://mlsanity.com/paper/2311.13230/) (2023)
- [VideoHallucer: Evaluating Intrinsic and Extrinsic Hallucinations in Large Video-Language Models](https://mlsanity.com/paper/2406.16338/) (2024)
- [ChatGPT as a Factual Inconsistency Evaluator for Text Summarization](https://mlsanity.com/paper/2303.15621/) (2023)
- [Mitigating Fine-Grained Hallucination by Fine-Tuning Large Vision-Language Models with Caption Rewrites](https://mlsanity.com/paper/2312.01701/) (2023)
- [A Survey on Dialogue Summarization: Recent Advances and New Frontiers](https://mlsanity.com/paper/2107.03175/) (2021)
- [Hallucination Detection: Robustly Discerning Reliable Answers in Large Language Models](https://mlsanity.com/paper/2407.04121/) (2024)
- [Is Stack Overflow Obsolete? An Empirical Study of the Characteristics of ChatGPT Answers to Stack Overflow Questions](https://mlsanity.com/paper/2308.02312/) (2023)
- [Correcting Diverse Factual Errors in Abstractive Summarization via Post-Editing and Language Model Infilling](https://mlsanity.com/paper/2210.12378/) (2022)
- [Treble Counterfactual VLMs: A Causal Approach to Hallucination](https://mlsanity.com/paper/2503.06169/) (2025)
- [AI Hallucinations: A Misnomer Worth Clarifying](https://mlsanity.com/paper/2401.06796/) (2024)
- [Confidence Under the Hood: An Investigation into the Confidence-Probability Alignment in Large Language Models](https://mlsanity.com/paper/2405.16282/) (2024)
- [Teaching Language Models to Hallucinate Less with Synthetic Tasks](https://mlsanity.com/paper/2310.06827/) (2023)
- [MQAG: Multiple-choice Question Answering and Generation for Assessing Information Consistency in Summarization](https://mlsanity.com/paper/2301.12307/) (2023)
- [KnowHalu: Hallucination Detection via Multi-Form Knowledge Based Factual Checking](https://mlsanity.com/paper/2404.02935/) (2024)
- [Are Large Language Models Reliable Judges? A Study on the Factuality Evaluation Capabilities of LLMs](https://mlsanity.com/paper/2311.00681/) (2023)
- [Mitigating Large Language Model Hallucination with Faithful Finetuning](https://mlsanity.com/paper/2406.11267/) (2024)
- [Small Language Model Can Self-correct](https://mlsanity.com/paper/2401.07301/) (2024)
- [CodeMirage: Hallucinations in Code Generated by Large Language Models](https://mlsanity.com/paper/2408.08333/) (2024)

## Abstract (arXiv)

Recently, various neural encoder-decoder models pioneered by Seq2Seq framework have been proposed to achieve the goal of generating more abstractive summaries by learning to map input text to output text. At a high level, such neural models can freely generate summaries without any constraint on the words or phrases used. Moreover, their format is closer to human-edited summaries and output is more readable and fluent. However, the neural model's abstraction ability is a double-edged sword. A commonly observed problem with the generated summaries is the distortion or fabrication of factual information in the article. This inconsistency between the original text and the summary has caused various concerns over its applicability, and the previous evaluation methods of text summarization are not suitable for this issue. In response to the above problems, the current research direction is predominantly divided into two categories, one is to design fact-aware evaluation metrics to select outputs without factual inconsistency errors, and the other is to develop new summarization systems towards factual consistency. In this survey, we focus on presenting a comprehensive review of these fact-specific evaluation methods and text summarization models.