# Knowledge Graph-Augmented Abstractive Summarization with Semantic-Driven Cloze Reward

arXiv: https://arxiv.org/abs/2005.01159 · HTML: https://mlsanity.com/paper/2005.01159/

## Key facts

- Cited by 66 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 3 May 2020.

## Built on by (most cited first, 30 of 66)

- [Survey of Hallucination in Natural Language Generation](https://mlsanity.com/paper/2202.03629/) (2022)
- [Trustworthy LLMs: a Survey and Guideline for Evaluating Large Language Models' Alignment](https://mlsanity.com/paper/2308.05374/) (2023)
- [Factuality Enhanced Language Models for Open-Ended Text Generation](https://mlsanity.com/paper/2206.04624/) (2022)
- [CLIFF: Contrastive Learning for Improving Faithfulness and Factuality in Abstractive Summarization](https://mlsanity.com/paper/2109.09209/) (2021)
- [Language Generation Models Can Cause Harm: So What Can We Do About It? An Actionable Survey](https://mlsanity.com/paper/2210.07700/) (2022)
- [The Factual Inconsistency Problem in Abstractive Text Summarization: A Survey](https://mlsanity.com/paper/2104.14839/) (2021)
- [Do Androids Know They're Only Dreaming of Electric Sheep?](https://mlsanity.com/paper/2312.17249/) (2023)
- [SummIt: Iterative Text Summarization via ChatGPT](https://mlsanity.com/paper/2305.14835/) (2023)
- [A Systematic Survey of Text Summarization: From Statistical Methods to Large Language Models](https://mlsanity.com/paper/2406.11289/) (2024)
- [Detecting and Mitigating Hallucinations in Multilingual Summarisation](https://mlsanity.com/paper/2305.13632/) (2023)
- [Re-Examining System-Level Correlations of Automatic Summarization Evaluation Metrics](https://mlsanity.com/paper/2204.10216/) (2022)
- [Structure-Aware Abstractive Conversation Summarization via Discourse and Action Graphs](https://mlsanity.com/paper/2104.08400/) (2021)
- [Learning to Revise References for Faithful Summarization](https://mlsanity.com/paper/2204.10290/) (2022)
- [Faithfulness in Natural Language Generation: A Systematic Survey of Analysis, Evaluation and Optimization Methods](https://mlsanity.com/paper/2203.05227/) (2022)
- [Graph Enhanced Contrastive Learning for Radiology Findings Summarization](https://mlsanity.com/paper/2204.00203/) (2022)
- [RetGen: A Joint framework for Retrieval and Grounded Text Generation Modeling](https://mlsanity.com/paper/2105.06597/) (2021)
- [An Overview on Language Models: Recent Developments and Outlook](https://mlsanity.com/paper/2303.05759/) (2023)
- [RECAP: Towards Precise Radiology Report Generation via Dynamic Disease Progression Reasoning](https://mlsanity.com/paper/2310.13864/) (2023)
- [BASS: Boosting Abstractive Summarization with Unified Semantic Graph](https://mlsanity.com/paper/2105.12041/) (2021)
- [RetrievalSum: A Retrieval Enhanced Framework for Abstractive Summarization](https://mlsanity.com/paper/2109.07943/) (2021)
- [Word Graph Guided Summarization for Radiology Findings](https://mlsanity.com/paper/2112.09925/) (2021)
- [Controllable Summarization with Constrained Markov Decision Process](https://mlsanity.com/paper/2108.03405/) (2021)
- [Enhancing Scientific Papers Summarization with Citation Graph](https://mlsanity.com/paper/2104.03057/) (2021)
- [Recent Advances in Neural Text Generation: A Task-Agnostic Survey](https://mlsanity.com/paper/2203.03047/) (2022)
- [NEREL: A Russian Dataset with Nested Named Entities, Relations and Events](https://mlsanity.com/paper/2108.13112/) (2021)
- [HETFORMER: Heterogeneous Transformer with Sparse Attention for Long-Text Extractive Summarization](https://mlsanity.com/paper/2110.06388/) (2021)
- [Deep Neural Approaches to Relation Triplets Extraction: A Comprehensive Survey](https://mlsanity.com/paper/2103.16929/) (2021)
- [Knowledge Infused Decoding](https://mlsanity.com/paper/2204.03084/) (2022)
- [Question-aware Transformer Models for Consumer Health Question Summarization](https://mlsanity.com/paper/2106.00219/) (2021)
- [Attention Head Masking for Inference Time Content Selection in Abstractive Summarization](https://mlsanity.com/paper/2104.02205/) (2021)

## Abstract (arXiv)

Sequence-to-sequence models for abstractive summarization have been studied extensively, yet the generated summaries commonly suffer from fabricated content, and are often found to be near-extractive. We argue that, to address these issues, the summarizer should acquire semantic interpretation over input, e.g., via structured representation, to allow the generation of more informative summaries. In this paper, we present ASGARD, a novel framework for Abstractive Summarization with Graph-Augmentation and semantic-driven RewarD. We propose the use of dual encoders---a sequential document encoder and a graph-structured encoder---to maintain the global context and local characteristics of entities, complementing each other. We further design a reward based on a multiple choice cloze test to drive the model to better capture entity interactions. Results show that our models produce significantly higher ROUGE scores than a variant without knowledge graph as input on both New York Times and CNN/Daily Mail datasets. We also obtain better or comparable performance compared to systems that are fine-tuned from large pretrained language models. Human judges further rate our model outputs as more informative and containing fewer unfaithful errors.