# Topic-Guided Abstractive Multi-Document Summarization

arXiv: https://arxiv.org/abs/2110.11207 · HTML: https://mlsanity.com/paper/2110.11207/

## Key facts

- Cited by 19 papers in the map (data through 14 Oct 2026).
- Builds on 22 of its 31 references that are in the map.
- First posted on arXiv 21 Oct 2021.

## Builds on (most cited first, 22 of 22)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [Graph Attention Networks](https://mlsanity.com/paper/1710.10903/) (2017)
- [Layer Normalization](https://mlsanity.com/paper/1607.06450/) (2016)
- [Don't Give Me the Details, Just the Summary! Topic-Aware Convolutional Neural Networks for Extreme Summarization](https://mlsanity.com/paper/1808.08745/) (2018)
- [Get To The Point: Summarization with Pointer-Generator Networks](https://mlsanity.com/paper/1704.04368/) (2017)
- [A Discourse-Aware Attention Model for Abstractive Summarization of Long Documents](https://mlsanity.com/paper/1804.05685/) (2018)
- [Multi-News: a Large-Scale Multi-Document Summarization Dataset and Abstractive Hierarchical Model](https://mlsanity.com/paper/1906.01749/) (2019)
- [Generating Wikipedia by Summarizing Long Sequences](https://mlsanity.com/paper/1801.10198/) (2018)
- [SummaRuNNer: A Recurrent Neural Network based Sequence Model for Extractive Summarization of Documents](https://mlsanity.com/paper/1611.04230/) (2016)
- [Extractive Summarization as Text Matching](https://mlsanity.com/paper/2004.08795/) (2020)
- [Bottom-Up Abstractive Summarization](https://mlsanity.com/paper/1808.10792/) (2018)
- [Autoencoding Variational Inference For Topic Models](https://mlsanity.com/paper/1703.01488/) (2017)
- [Hierarchical Transformers for Multi-Document Summarization](https://mlsanity.com/paper/1905.13164/) (2019)
- [HIBERT: Document Level Pre-training of Hierarchical Bidirectional Transformers for Document Summarization](https://mlsanity.com/paper/1905.06566/) (2019)
- [Heterogeneous Graph Neural Networks for Extractive Document Summarization](https://mlsanity.com/paper/2004.12393/) (2020)
- [Discovering Discrete Latent Topics with Neural Variational Inference](https://mlsanity.com/paper/1706.00359/) (2017)
- [BanditSum: Extractive Summarization as a Contextual Bandit](https://mlsanity.com/paper/1809.09672/) (2018)
- [Topic-Aware Neural Keyphrase Generation for Social Media Language](https://mlsanity.com/paper/1906.03889/) (2019)
- [Topic Memory Networks for Short Text Classification](https://mlsanity.com/paper/1809.03664/) (2018)
- [Enhancing Extractive Text Summarization with Topic-Aware Graph Neural Networks](https://mlsanity.com/paper/2010.06253/) (2020)
- [Neural Latent Extractive Document Summarization](https://mlsanity.com/paper/1808.07187/) (2018)

## Built on by (most cited first, 19 of 19)

- [LongT5: Efficient Text-To-Text Transformer for Long Sequences](https://mlsanity.com/paper/2112.07916/) (2021)
- [ChatGPT vs Human-authored Text: Insights into Controllable Text Summarization and Sentence Style Transfer](https://mlsanity.com/paper/2306.07799/) (2023)
- [HEGEL: Hypergraph Transformer for Long Document Summarization](https://mlsanity.com/paper/2210.04126/) (2022)
- [NeuS: Neutral Multi-News Summarization for Mitigating Framing Bias](https://mlsanity.com/paper/2204.04902/) (2022)
- [Summarizing Multiple Documents with Conversational Structure for Meta-Review Generation](https://mlsanity.com/paper/2305.01498/) (2023)
- [Adaptive and Personalized Exercise Generation for Online Language Learning](https://mlsanity.com/paper/2306.02457/) (2023)
- [DeTiME: Diffusion-Enhanced Topic Modeling using Encoder-decoder based LLM](https://mlsanity.com/paper/2310.15296/) (2023)
- [GRETEL: Graph Contrastive Topic Enhanced Language Model for Long Document Extractive Summarization](https://mlsanity.com/paper/2208.09982/) (2022)
- [Contrastive Hierarchical Discourse Graph for Scientific Document Summarization](https://mlsanity.com/paper/2306.00177/) (2023)
- [A Topic-aware Summarization Framework with Different Modal Side Information](https://mlsanity.com/paper/2305.11503/) (2023)
- [Compressed Heterogeneous Graph for Abstractive Multi-Document Summarization](https://mlsanity.com/paper/2303.06565/) (2023)
- [A Hierarchical Encoding-Decoding Scheme for Abstractive Multi-document Summarization](https://mlsanity.com/paper/2305.08503/) (2023)
- [Multi-document Summarization through Multi-document Event Relation Graph Reasoning in LLMs: a case study in Framing Bias Mitigation](https://mlsanity.com/paper/2506.12978/) (2025)
- [Controllable Topic-Focused Abstractive Summarization](https://mlsanity.com/paper/2311.06724/) (2023)
- [Learning Cluster Patterns for Abstractive Summarization](https://mlsanity.com/paper/2202.10967/) (2022)
- [Moving beyond word lists: towards abstractive topic labels for human-like topics of scientific documents](https://mlsanity.com/paper/2211.05599/) (2022)
- [Tracking Brand-Associated Polarity-Bearing Topics in User Reviews](https://mlsanity.com/paper/2301.07183/) (2023)
- [A Unified Retrieval Framework with Document Ranking and EDU Filtering for Multi-document Summarization](https://mlsanity.com/paper/2504.16711/) (2025)
- [Topic-Guided Reinforcement Learning with LLMs for Enhancing Multi-Document Summarization](https://mlsanity.com/paper/2509.09852/) (2025)

## Abstract (arXiv)

A critical point of multi-document summarization (MDS) is to learn the relations among various documents. In this paper, we propose a novel abstractive MDS model, in which we represent multiple documents as a heterogeneous graph, taking semantic nodes of different granularities into account, and then apply a graph-to-sequence framework to generate summaries. Moreover, we employ a neural topic model to jointly discover latent topics that can act as cross-document semantic units to bridge different documents and provide global information to guide the summary generation. Since topic extraction can be viewed as a special type of summarization that "summarizes" texts into a more abstract format, i.e., a topic distribution, we adopt a multi-task learning strategy to jointly train the topic and summarization module, allowing the promotion of each other. Experimental results on the Multi-News dataset demonstrate that our model outperforms previous state-of-the-art MDS models on both Rouge metrics and human evaluation, meanwhile learns high-quality topics.