# Mind The Facts: Knowledge-Boosted Coherent Abstractive Text Summarization

arXiv: https://arxiv.org/abs/2006.15435 · HTML: https://mlsanity.com/paper/2006.15435/

## Key facts

- Cited by 13 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 27 Jun 2020.

## Built on by (most cited first, 13 of 13)

- [Survey of Hallucination in Natural Language Generation](https://mlsanity.com/paper/2202.03629/) (2022)
- [The Factual Inconsistency Problem in Abstractive Text Summarization: A Survey](https://mlsanity.com/paper/2104.14839/) (2021)
- [Dict-BERT: Enhancing Language Model Pre-training with Dictionary](https://mlsanity.com/paper/2110.06490/) (2021)
- [Hierarchical Learning for Generation with Long Source Sequences](https://mlsanity.com/paper/2104.07545/) (2021)
- [Faithful to the Document or to the World? Mitigating Hallucinations via Entity-linked Knowledge in Abstractive Summarization](https://mlsanity.com/paper/2204.13761/) (2022)
- [Contrastive Error Attribution for Finetuned Language Models](https://mlsanity.com/paper/2212.10722/) (2022)
- [Challenges in Domain-Specific Abstractive Summarization and How to Overcome them](https://mlsanity.com/paper/2307.00963/) (2023)
- [Sequence-Level Certainty Reduces Hallucination In Knowledge-Grounded Dialogue Generation](https://mlsanity.com/paper/2310.18794/) (2023)
- [Descartes: Generating Short Descriptions of Wikipedia Articles](https://mlsanity.com/paper/2205.10012/) (2022)
- [Entity-driven Fact-aware Abstractive Summarization of Biomedical Literature](https://mlsanity.com/paper/2203.15959/) (2022)
- [Topics as Entity Clusters: Entity-based Topics from Large Language Models and Graph Neural Networks](https://mlsanity.com/paper/2301.02458/) (2023)
- [Knowledge-aware Document Summarization: A Survey of Knowledge, Embedding Methods and Architectures](https://mlsanity.com/paper/2204.11190/) (2022)
- [An entity-guided text summarization framework with relational heterogeneous graph neural network](https://mlsanity.com/paper/2302.03205/) (2023)

## Abstract (arXiv)

Neural models have become successful at producing abstractive summaries that are human-readable and fluent. However, these models have two critical shortcomings: they often don't respect the facts that are either included in the source article or are known to humans as commonsense knowledge, and they don't produce coherent summaries when the source article is long. In this work, we propose a novel architecture that extends Transformer encoder-decoder architecture in order to improve on these shortcomings. First, we incorporate entity-level knowledge from the Wikidata knowledge graph into the encoder-decoder architecture. Injecting structural world knowledge from Wikidata helps our abstractive summarization model to be more fact-aware. Second, we utilize the ideas used in Transformer-XL language model in our proposed encoder-decoder architecture. This helps our model with producing coherent summaries even when the source article is long. We test our model on CNN/Daily Mail summarization dataset and show improvements on ROUGE scores over the baseline Transformer model. We also include model predictions for which our model accurately conveys the facts, while the baseline Transformer model doesn't.