# How "Multi" is Multi-Document Summarization?

arXiv: https://arxiv.org/abs/2210.12688 · HTML: https://mlsanity.com/paper/2210.12688/

## Key facts

- Cited by 8 papers in the map (data through 14 Oct 2026).
- Builds on 15 of its 26 references that are in the map.
- First posted on arXiv 23 Oct 2022.

## Builds on (most cited first, 15 of 15)

- [Get To The Point: Summarization with Pointer-Generator Networks](https://mlsanity.com/paper/1704.04368/) (2017)
- [On Faithfulness and Factuality in Abstractive Summarization](https://mlsanity.com/paper/2005.00661/) (2020)
- [Evaluating the Factual Consistency of Abstractive Text Summarization](https://mlsanity.com/paper/1910.12840/) (2019)
- [Multi-News: a Large-Scale Multi-Document Summarization Dataset and Abstractive Hierarchical Model](https://mlsanity.com/paper/1906.01749/) (2019)
- [Newsroom: A Dataset of 1.3 Million Summaries with Diverse Extractive Strategies](https://mlsanity.com/paper/1804.11283/) (2018)
- [QuestEval: Summarization Asks for Fact-based Evaluation](https://mlsanity.com/paper/2103.12693/) (2021)
- [MS2: Multi-Document Summarization of Medical Studies](https://mlsanity.com/paper/2104.06486/) (2021)
- [A Large-Scale Multi-Document Summarization Dataset from the Wikipedia Current Events Portal](https://mlsanity.com/paper/2005.10070/) (2020)
- [PRIMERA: Pyramid-based Masked Sentence Pre-training for Multi-document Summarization](https://mlsanity.com/paper/2110.08499/) (2021)
- [Adapting the Neural Encoder-Decoder Framework from Single to Multi-Document Summarization](https://mlsanity.com/paper/1808.06218/) (2018)
- [Earlier Isn't Always Better: Sub-aspect Analysis on Corpus and System Biases in Summarization](https://mlsanity.com/paper/1908.11723/) (2019)
- [Multi-document Summarization with Maximal Marginal Relevance-guided Reinforcement Learning](https://mlsanity.com/paper/2010.00117/) (2020)
- [Summary-Source Proposition-level Alignment: Task, Datasets and Supervised Baseline](https://mlsanity.com/paper/2009.00590/) (2020)
- [Systematically Exploring Redundancy Reduction in Summarizing Long Documents](https://mlsanity.com/paper/2012.00052/) (2020)
- [Analyzing Sentence Fusion in Abstractive Summarization](https://mlsanity.com/paper/1910.00203/) (2019)

## Built on by (most cited first, 8 of 8)

- [Automated Metrics for Medical Multi-Document Summarization Disagree with Human Evaluations](https://mlsanity.com/paper/2305.13693/) (2023)
- [Do Multi-Document Summarization Models Synthesize?](https://mlsanity.com/paper/2301.13844/) (2023)
- [LLM Based Multi-Document Summarization Exploiting Main-Event Biased Monotone Submodular Content Extraction](https://mlsanity.com/paper/2310.03414/) (2023)
- [Multi-document Summarization: A Comparative Evaluation](https://mlsanity.com/paper/2309.04951/) (2023)
- [A Hierarchical Encoding-Decoding Scheme for Abstractive Multi-document Summarization](https://mlsanity.com/paper/2305.08503/) (2023)
- [Controllable Multi-document Summarization: Coverage & Coherence Intuitive Policy with Large Language Model Based Rewards](https://mlsanity.com/paper/2310.03473/) (2023)
- [Disentangling Specificity for Abstractive Multi-document Summarization](https://mlsanity.com/paper/2406.00005/) (2024)
- [A Unified Retrieval Framework with Document Ranking and EDU Filtering for Multi-document Summarization](https://mlsanity.com/paper/2504.16711/) (2025)

## Abstract (arXiv)

The task of multi-document summarization (MDS) aims at models that, given multiple documents as input, are able to generate a summary that combines disperse information, originally spread across these documents. Accordingly, it is expected that both reference summaries in MDS datasets, as well as system summaries, would indeed be based on such dispersed information. In this paper, we argue for quantifying and assessing this expectation. To that end, we propose an automated measure for evaluating the degree to which a summary is ``disperse'', in the sense of the number of source documents needed to cover its content. We apply our measure to empirically analyze several popular MDS datasets, with respect to their reference summaries, as well as the output of state-of-the-art systems. Our results show that certain MDS datasets barely require combining information from multiple documents, where a single document often covers the full summary content. Overall, we advocate using our metric for assessing and improving the degree to which summarization datasets require combining multi-document information, and similarly how summarization models actually meet this challenge. Our code is available in https://github.com/ariecattan/multi_mds.