# Constrained Abstractive Summarization: Preserving Factual Consistency with Constrained Generation

arXiv: https://arxiv.org/abs/2010.12723 · HTML: https://mlsanity.com/paper/2010.12723/

## Key facts

- Cited by 26 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 24 Oct 2020.

## Built on by (most cited first, 26 of 26)

- [Annotating and Modeling Fine-grained Factuality in Summarization](https://mlsanity.com/paper/2104.04302/) (2021)
- [Language Generation Models Can Cause Harm: So What Can We Do About It? An Actionable Survey](https://mlsanity.com/paper/2210.07700/) (2022)
- [The Factual Inconsistency Problem in Abstractive Text Summarization: A Survey](https://mlsanity.com/paper/2104.14839/) (2021)
- [Planning with Learned Entity Prompts for Abstractive Summarization](https://mlsanity.com/paper/2104.07606/) (2021)
- [Mutual Information Alleviates Hallucinations in Abstractive Summarization](https://mlsanity.com/paper/2210.13210/) (2022)
- [Faithfulness in Natural Language Generation: A Systematic Survey of Analysis, Evaluation and Optimization Methods](https://mlsanity.com/paper/2203.05227/) (2022)
- [Don't Say What You Don't Know: Improving the Consistency of Abstractive Summarization by Constraining Beam Search](https://mlsanity.com/paper/2203.08436/) (2022)
- [Faithful to the Document or to the World? Mitigating Hallucinations via Entity-linked Knowledge in Abstractive Summarization](https://mlsanity.com/paper/2204.13761/) (2022)
- ["Why is this misleading?": Detecting News Headline Hallucinations with Explanations](https://mlsanity.com/paper/2302.05852/) (2023)
- [Mitigating Covertly Unsafe Text within Natural Language Systems](https://mlsanity.com/paper/2210.09306/) (2022)
- [Knowledge Infused Decoding](https://mlsanity.com/paper/2204.03084/) (2022)
- [CaPE: Contrastive Parameter Ensembling for Reducing Hallucination in Abstractive Summarization](https://mlsanity.com/paper/2110.07166/) (2021)
- [Expository Text Generation: Imitate, Retrieve, Paraphrase](https://mlsanity.com/paper/2305.03276/) (2023)
- [Key-Element-Informed sLLM Tuning for Document Summarization](https://mlsanity.com/paper/2406.04625/) (2024)
- [Probing Factually Grounded Content Transfer with Factual Ablation](https://mlsanity.com/paper/2203.10133/) (2022)
- [Mitigating Hallucination in Abstractive Summarization with Domain-Conditional Mutual Information](https://mlsanity.com/paper/2404.09480/) (2024)
- [Controlling the Focus of Pretrained Language Generation Models](https://mlsanity.com/paper/2203.01146/) (2022)
- [PCFG-based Natural Language Interface Improves Generalization for Controlled Text Generation](https://mlsanity.com/paper/2210.07431/) (2022)
- [Improving Factual Consistency of Abstractive Summarization on Customer Feedback](https://mlsanity.com/paper/2106.16188/) (2021)
- [Named Entity Inclusion in Abstractive Text Summarization](https://mlsanity.com/paper/2307.02570/) (2023)
- [uMedSum: A Unified Framework for Advancing Medical Abstractive Summarization](https://mlsanity.com/paper/2408.12095/) (2024)
- [InFillmore: Frame-Guided Language Generation with Bidirectional Context](https://mlsanity.com/paper/2103.04941/) (2021)
- [Extract, Denoise and Enforce: Evaluating and Improving Concept Preservation for Text-to-Text Generation](https://mlsanity.com/paper/2104.08724/) (2021)
- [Leveraging Key Information Modeling to Improve Less-Data Constrained News Headline Generation via Duality Fine-Tuning](https://mlsanity.com/paper/2210.04473/) (2022)
- [Neural models for Factual Inconsistency Classification with Explanations](https://mlsanity.com/paper/2306.08872/) (2023)
- [Text Fact Transfer](https://mlsanity.com/paper/2310.14486/) (2023)

## Abstract (arXiv)

Despite significant progress, state-of-the-art abstractive summarization methods are still prone to hallucinate content inconsistent with the source document. In this paper, we propose Constrained Abstractive Summarization (CAS), a general setup that preserves the factual consistency of abstractive summarization by specifying tokens as constraints that must be present in the summary. We adopt lexically constrained decoding, a technique generally applicable to autoregressive generative models, to fulfill CAS and conduct experiments in two scenarios: (1) automatic summarization without human involvement, where keyphrases are extracted from the source document and used as constraints; (2) human-guided interactive summarization, where human feedback in the form of manual constraints are used to guide summary generation. Automatic and human evaluations on two benchmark datasets demonstrate that CAS improves both lexical overlap (ROUGE) and factual consistency of abstractive summarization. In particular, we observe up to 13.8 ROUGE-2 gains when only one manual constraint is used in interactive summarization.