# Abstractive Summarization Guided by Latent Hierarchical Document Structure

arXiv: https://arxiv.org/abs/2211.09458 · HTML: https://mlsanity.com/paper/2211.09458/

## Key facts

- Cited by 4 papers in the map (data through 14 Oct 2026).
- Builds on 32 of its 60 references that are in the map.
- First posted on arXiv 17 Nov 2022.

## Builds on (most cited first, 30 of 32)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [BART: Denoising Sequence-to-Sequence Pre-training for Natural Language Generation, Translation, and Comprehension](https://mlsanity.com/paper/1910.13461/) (2019)
- [BERTScore: Evaluating Text Generation with BERT](https://mlsanity.com/paper/1904.09675/) (2019)
- [Don't Give Me the Details, Just the Summary! Topic-Aware Convolutional Neural Networks for Extreme Summarization](https://mlsanity.com/paper/1808.08745/) (2018)
- [Get To The Point: Summarization with Pointer-Generator Networks](https://mlsanity.com/paper/1704.04368/) (2017)
- [Convolutional Neural Networks for Sentence Classification](https://mlsanity.com/paper/1408.5882/) (2014)
- [Teaching Machines to Read and Comprehend](https://mlsanity.com/paper/1506.03340/) (2015)
- [Effective Approaches to Attention-based Neural Machine Translation](https://mlsanity.com/paper/1508.04025/) (2015)
- [Text Summarization with Pretrained Encoders](https://mlsanity.com/paper/1908.08345/) (2019)
- [A Discourse-Aware Attention Model for Abstractive Summarization of Long Documents](https://mlsanity.com/paper/1804.05685/) (2018)
- [A Neural Attention Model for Abstractive Sentence Summarization](https://mlsanity.com/paper/1509.00685/) (2015)
- [Generating Wikipedia by Summarizing Long Sequences](https://mlsanity.com/paper/1801.10198/) (2018)
- [SummaRuNNer: A Recurrent Neural Network based Sequence Model for Extractive Summarization of Documents](https://mlsanity.com/paper/1611.04230/) (2016)
- [Adaptively Sparse Transformers](https://mlsanity.com/paper/1909.00015/) (2019)
- [GSum: A General Framework for Guided Neural Abstractive Summarization](https://mlsanity.com/paper/2010.08014/) (2020)
- [Hallucinated but Factual! Inspecting the Factuality of Hallucinations in Abstractive Summarization](https://mlsanity.com/paper/2109.09784/) (2021)
- [Heterogeneous Graph Neural Networks for Extractive Document Summarization](https://mlsanity.com/paper/2004.12393/) (2020)
- [On Extractive and Abstractive Neural Document Summarization with Transformer Language Models](https://mlsanity.com/paper/1909.03186/) (2019)
- [Knowledge Graph-Augmented Abstractive Summarization with Semantic-Driven Cloze Reward](https://mlsanity.com/paper/2005.01159/) (2020)
- [What Have We Achieved on Text Summarization?](https://mlsanity.com/paper/2010.04529/) (2020)
- [A C-LSTM Neural Network for Text Classification](https://mlsanity.com/paper/1511.08630/) (2015)
- [Topic-aware Pointer-Generator Networks for Summarizing Spoken Conversations](https://mlsanity.com/paper/1910.01335/) (2019)
- [Sparse Attention with Linear Units](https://mlsanity.com/paper/2104.07012/) (2021)
- [Factual Consistency Evaluation for Text Summarization via Counterfactual Estimation](https://mlsanity.com/paper/2108.13134/) (2021)
- [Scoring Sentence Singletons and Pairs for Abstractive Summarization](https://mlsanity.com/paper/1906.00077/) (2019)
- [Hierarchical Learning for Generation with Long Source Sequences](https://mlsanity.com/paper/2104.07545/) (2021)
- [Guided Neural Language Generation for Abstractive Summarization using Abstract Meaning Representation](https://mlsanity.com/paper/1808.09160/) (2018)
- [Enhancing Extractive Text Summarization with Topic-Aware Graph Neural Networks](https://mlsanity.com/paper/2010.06253/) (2020)
- [BASS: Boosting Abstractive Summarization with Unified Semantic Graph](https://mlsanity.com/paper/2105.12041/) (2021)
- [Structure-Infused Copy Mechanisms for Abstractive Summarization](https://mlsanity.com/paper/1806.05658/) (2018)

## Built on by (most cited first, 4 of 4)

- [Detecting and Mitigating Hallucinations in Multilingual Summarisation](https://mlsanity.com/paper/2305.13632/) (2023)
- [Abstractive Text Summarization: State of the Art, Challenges, and Improvements](https://mlsanity.com/paper/2409.02413/) (2024)
- [Think While You Write: Hypothesis Verification Promotes Faithful Knowledge-to-Text Generation](https://mlsanity.com/paper/2311.09467/) (2023)
- [Seg2Act: Global Context-aware Action Generation for Document Logical Structuring](https://mlsanity.com/paper/2410.06802/) (2024)

## Abstract (arXiv)

Sequential abstractive neural summarizers often do not use the underlying structure in the input article or dependencies between the input sentences. This structure is essential to integrate and consolidate information from different parts of the text. To address this shortcoming, we propose a hierarchy-aware graph neural network (HierGNN) which captures such dependencies through three main steps: 1) learning a hierarchical document structure through a latent structure tree learned by a sparse matrix-tree computation; 2) propagating sentence information over this structure using a novel message-passing node propagation mechanism to identify salient information; 3) using graph-level attention to concentrate the decoder on salient information. Experiments confirm HierGNN improves strong sequence models such as BART, with a 0.55 and 0.75 margin in average ROUGE-1/2/L for CNN/DM and XSum. Further human evaluation demonstrates that summaries produced by our model are more relevant and less redundant than the baselines, into which HierGNN is incorporated. We also find HierGNN synthesizes summaries by fusing multiple source sentences more, rather than compressing a single source sentence, and that it processes long inputs more effectively.