# Multi-document Summarization: A Comparative Evaluation

arXiv: https://arxiv.org/abs/2309.04951 · HTML: https://mlsanity.com/paper/2309.04951/

## Key facts

- Cited by 3 papers in the map (data through 14 Oct 2026).
- Builds on 18 of its 33 references that are in the map.
- First posted on arXiv 10 Sep 2023.

## Builds on (most cited first, 18 of 18)

- [Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformer](https://mlsanity.com/paper/1910.10683/) (2019)
- [BART: Denoising Sequence-to-Sequence Pre-training for Natural Language Generation, Translation, and Comprehension](https://mlsanity.com/paper/1910.13461/) (2019)
- [Longformer: The Long-Document Transformer](https://mlsanity.com/paper/2004.05150/) (2020)
- [Big Bird: Transformers for Longer Sequences](https://mlsanity.com/paper/2007.14062/) (2020)
- [PEGASUS: Pre-training with Extracted Gap-sentences for Abstractive Summarization](https://mlsanity.com/paper/1912.08777/) (2019)
- [Text Summarization with Pretrained Encoders](https://mlsanity.com/paper/1908.08345/) (2019)
- [Multi-News: a Large-Scale Multi-Document Summarization Dataset and Abstractive Hierarchical Model](https://mlsanity.com/paper/1906.01749/) (2019)
- [Generating Wikipedia by Summarizing Long Sequences](https://mlsanity.com/paper/1801.10198/) (2018)
- [Newsroom: A Dataset of 1.3 Million Summaries with Diverse Extractive Strategies](https://mlsanity.com/paper/1804.11283/) (2018)
- [WikiHow: A Large Scale Text Summarization Dataset](https://mlsanity.com/paper/1810.09305/) (2018)
- [Multi-XScience: A Large-scale Dataset for Extreme Multi-document Summarization of Scientific Articles](https://mlsanity.com/paper/2010.14235/) (2020)
- [MS2: Multi-Document Summarization of Medical Studies](https://mlsanity.com/paper/2104.06486/) (2021)
- [PRIMERA: Pyramid-based Masked Sentence Pre-training for Multi-document Summarization](https://mlsanity.com/paper/2110.08499/) (2021)
- [Multi-document Summarization via Deep Learning Techniques: A Survey](https://mlsanity.com/paper/2011.04843/) (2020)
- [Generating a Structured Summary of Numerous Academic Papers: Dataset and Method](https://mlsanity.com/paper/2302.04580/) (2023)
- [Do Multi-Document Summarization Models Synthesize?](https://mlsanity.com/paper/2301.13844/) (2023)
- [How "Multi" is Multi-Document Summarization?](https://mlsanity.com/paper/2210.12688/) (2022)
- [Dependency Structure for News Document Summarization](https://mlsanity.com/paper/2109.11199/) (2021)

## Built on by (most cited first, 3 of 3)

- [MDCure: A Scalable Pipeline for Multi-Document Instruction-Following](https://mlsanity.com/paper/2410.23463/) (2024)
- [Document Intelligence in the Era of Large Language Models: A Survey](https://mlsanity.com/paper/2510.13366/) (2025)
- [M2DS: Multilingual Dataset for Multi-document Summarisation](https://mlsanity.com/paper/2407.12336/) (2024)

## Abstract (arXiv)

This paper is aimed at evaluating state-of-the-art models for Multi-document Summarization (MDS) on different types of datasets in various domains and investigating the limitations of existing models to determine future research directions. To address this gap, we conducted an extensive literature review to identify state-of-the-art models and datasets. We analyzed the performance of PRIMERA and PEGASUS models on BigSurvey-MDS and MS2 datasets, which posed unique challenges due to their varied domains. Our findings show that the General-Purpose Pre-trained Model LED outperforms PRIMERA and PEGASUS on the MS2 dataset. We used the ROUGE score as a performance metric to evaluate the identified models on different datasets. Our study provides valuable insights into the models' strengths and weaknesses, as well as their applicability in different domains. This work serves as a reference for future MDS research and contributes to the development of accurate and robust models which can be utilized on demanding datasets with academically and/or scientifically complex data as well as generalized, relatively simple datasets.