# Abstractive Summarization of Large Document Collections Using GPT

arXiv: https://arxiv.org/abs/2310.05690 · HTML: https://mlsanity.com/paper/2310.05690/

## Key facts

- Cited by 6 papers in the map (data through 14 Oct 2026).
- Builds on 21 of its 94 references that are in the map.
- First posted on arXiv 9 Oct 2023.

## Builds on (most cited first, 21 of 21)

- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [Language Models are Few-Shot Learners](https://mlsanity.com/paper/2005.14165/) (2020)
- [LLaMA: Open and Efficient Foundation Language Models](https://mlsanity.com/paper/2302.13971/) (2023)
- [Sentence-BERT: Sentence Embeddings using Siamese BERT-Networks](https://mlsanity.com/paper/1908.10084/) (2019)
- [BART: Denoising Sequence-to-Sequence Pre-training for Natural Language Generation, Translation, and Comprehension](https://mlsanity.com/paper/1910.13461/) (2019)
- [UMAP: Uniform Manifold Approximation and Projection for Dimension Reduction](https://mlsanity.com/paper/1802.03426/) (2018)
- [Billion-scale similarity search with GPUs](https://mlsanity.com/paper/1702.08734/) (2017)
- [BLOOM: A 176B-Parameter Open-Access Multilingual Language Model](https://mlsanity.com/paper/2211.05100/) (2022)
- [PEGASUS: Pre-training with Extracted Gap-sentences for Abstractive Summarization](https://mlsanity.com/paper/1912.08777/) (2019)
- [Abstractive Text Summarization Using Sequence-to-Sequence RNNs and Beyond](https://mlsanity.com/paper/1602.06023/) (2016)
- [A Neural Attention Model for Abstractive Sentence Summarization](https://mlsanity.com/paper/1509.00685/) (2015)
- [MASS: Masked Sequence to Sequence Pre-training for Language Generation](https://mlsanity.com/paper/1905.02450/) (2019)
- [Crowdsourcing a Word-Emotion Association Lexicon](https://mlsanity.com/paper/1308.6297/) (2013)
- [BRIO: Bringing Order to Abstractive Summarization](https://mlsanity.com/paper/2203.16804/) (2022)
- [Deep Learning for Sentiment Analysis : A Survey](https://mlsanity.com/paper/1801.07883/) (2018)
- [Stance and Sentiment in Tweets](https://mlsanity.com/paper/1605.01655/) (2016)
- [Toward Abstractive Summarization Using Semantic Representations](https://mlsanity.com/paper/1805.10399/) (2018)
- [Sentiment Analysis: Automatically Detecting Valence, Emotions, and Other Affectual States from Text](https://mlsanity.com/paper/2005.11882/) (2020)
- [Uniform Manifold Approximation and Projection (UMAP) and its Variants: Tutorial and Survey](https://mlsanity.com/paper/2109.02508/) (2021)
- [Momentum Calibration for Text Generation](https://mlsanity.com/paper/2212.04257/) (2022)
- [A Survey on Neural Abstractive Summarization Methods and Factual Consistency of Summarization](https://mlsanity.com/paper/2204.09519/) (2022)

## Built on by (most cited first, 6 of 6)

- [Document Intelligence in the Era of Large Language Models: A Survey](https://mlsanity.com/paper/2510.13366/) (2025)
- [Assisting humans in complex comparisons: automated information comparison at scale](https://mlsanity.com/paper/2404.04351/) (2024)
- [Automated Multi-Language to English Machine Translation Using Generative Pre-Trained Transformers](https://mlsanity.com/paper/2404.14680/) (2024)
- [DaVinci at SemEval-2024 Task 9: Few-shot prompting GPT-3.5 for Unconventional Reasoning](https://mlsanity.com/paper/2405.11559/) (2024)
- [Large Language Model and Formal Concept Analysis: a comparative study for Topic Modeling](https://mlsanity.com/paper/2602.01933/) (2026)
- [Althea: The Fact-Checking--Metalearning Tradeoff in AI-Assisted Verification](https://mlsanity.com/paper/2602.11161/) (2025)

## Abstract (arXiv)

This paper proposes a method of abstractive summarization designed to scale to document collections instead of individual documents. Our approach applies a combination of semantic clustering, document size reduction within topic clusters, semantic chunking of a cluster's documents, GPT-based summarization and concatenation, and a combined sentiment and text visualization of each topic to support exploratory data analysis. Statistical comparison of our results to existing state-of-the-art systems BART, BRIO, PEGASUS, and MoCa using ROGUE summary scores showed statistically equivalent performance with BART and PEGASUS on the CNN/Daily Mail test dataset, and with BART on the Gigaword test dataset. This finding is promising since we view document collection summarization as more challenging than individual document summarization. We conclude with a discussion of how issues of scale are