# Reducing Quantity Hallucinations in Abstractive Summarization

arXiv: https://arxiv.org/abs/2009.13312 · HTML: https://mlsanity.com/paper/2009.13312/

## Key facts

- Cited by 22 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 28 Sep 2020.

## Built on by (most cited first, 22 of 22)

- [Survey of Hallucination in Natural Language Generation](https://mlsanity.com/paper/2202.03629/) (2022)
- [Annotating and Modeling Fine-grained Factuality in Summarization](https://mlsanity.com/paper/2104.04302/) (2021)
- [Hallucinated but Factual! Inspecting the Factuality of Hallucinations in Abstractive Summarization](https://mlsanity.com/paper/2109.09784/) (2021)
- [Language Generation Models Can Cause Harm: So What Can We Do About It? An Actionable Survey](https://mlsanity.com/paper/2210.07700/) (2022)
- [Increasing Faithfulness in Knowledge-Grounded Dialogue with Controllable Features](https://mlsanity.com/paper/2107.06963/) (2021)
- [AI Hallucinations: A Misnomer Worth Clarifying](https://mlsanity.com/paper/2401.06796/) (2024)
- [Learning to Revise References for Faithful Summarization](https://mlsanity.com/paper/2204.10290/) (2022)
- [Faithfulness in Natural Language Generation: A Systematic Survey of Analysis, Evaluation and Optimization Methods](https://mlsanity.com/paper/2203.05227/) (2022)
- [Tailoring Language Generation Models under Total Variation Distance](https://mlsanity.com/paper/2302.13344/) (2023)
- [Discharge Summary Hospital Course Summarisation of In Patient Electronic Health Record Text with Clinical Concept Guided Deep Pre-Trained Transformer Models](https://mlsanity.com/paper/2211.07126/) (2022)
- [Investigating Hallucinations in Pruned Large Language Models for Abstractive Summarization](https://mlsanity.com/paper/2311.09335/) (2023)
- [Shortcomings of Question Answering Based Factuality Frameworks for Error Localization](https://mlsanity.com/paper/2210.06748/) (2022)
- [Hierarchical Summarization for Longform Spoken Dialog](https://mlsanity.com/paper/2108.09597/) (2021)
- [A Comprehensive Survey on Legal Summarization: Challenges and Future Directions](https://mlsanity.com/paper/2501.17830/) (2025)
- [Masked Summarization to Generate Factually Inconsistent Summaries for Improved Factual Consistency Checking](https://mlsanity.com/paper/2205.02035/) (2022)
- [Summary Explorer: Visualizing the State of the Art in Text Summarization](https://mlsanity.com/paper/2108.01879/) (2021)
- [How Optimal is Greedy Decoding for Extractive Question Answering?](https://mlsanity.com/paper/2108.05857/) (2021)
- [Improving abstractive summarization with energy-based re-ranking](https://mlsanity.com/paper/2210.15553/) (2022)
- [Towards Differential Relational Privacy and its use in Question Answering](https://mlsanity.com/paper/2203.16701/) (2022)
- [Survey on Abstractive Text Summarization: Dataset, Models, and Metrics](https://mlsanity.com/paper/2412.17165/) (2024)
- [Probing of Quantitative Values in Abstractive Summarization Models](https://mlsanity.com/paper/2210.00667/) (2022)
- [Is In-hospital Meta-information Useful for Abstractive Discharge Summary Generation?](https://mlsanity.com/paper/2303.06002/) (2023)

## Abstract (arXiv)

It is well-known that abstractive summaries are subject to hallucination---including material that is not supported by the original text. While summaries can be made hallucination-free by limiting them to general phrases, such summaries would fail to be very informative. Alternatively, one can try to avoid hallucinations by verifying that any specific entities in the summary appear in the original text in a similar context. This is the approach taken by our system, Herman. The system learns to recognize and verify quantity entities (dates, numbers, sums of money, etc.) in a beam-worth of abstractive summaries produced by state-of-the-art models, in order to up-rank those summaries whose quantity terms are supported by the original text. Experimental results demonstrate that the ROUGE scores of such up-ranked summaries have a higher Precision than summaries that have not been up-ranked, without a comparable loss in Recall, resulting in higher F1. Preliminary human evaluation of up-ranked vs. original summaries shows people's preference for the former.