# Summ^N: A Multi-Stage Summarization Framework for Long Input Dialogues and Documents

arXiv: https://arxiv.org/abs/2110.10150 · HTML: https://mlsanity.com/paper/2110.10150/

## Key facts

- Cited by 65 papers in the map (data through 14 Oct 2026).
- Builds on 33 of its 51 references that are in the map.
- First posted on arXiv 16 Oct 2021.

## Builds on (most cited first, 30 of 33)

- [Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformer](https://mlsanity.com/paper/1910.10683/) (2019)
- [BART: Denoising Sequence-to-Sequence Pre-training for Natural Language Generation, Translation, and Comprehension](https://mlsanity.com/paper/1910.13461/) (2019)
- [Longformer: The Long-Document Transformer](https://mlsanity.com/paper/2004.05150/) (2020)
- [Big Bird: Transformers for Longer Sequences](https://mlsanity.com/paper/2007.14062/) (2020)
- [Reformer: The Efficient Transformer](https://mlsanity.com/paper/2001.04451/) (2020)
- [Get To The Point: Summarization with Pointer-Generator Networks](https://mlsanity.com/paper/1704.04368/) (2017)
- [Hierarchical Neural Story Generation](https://mlsanity.com/paper/1805.04833/) (2018)
- [PEGASUS: Pre-training with Extracted Gap-sentences for Abstractive Summarization](https://mlsanity.com/paper/1912.08777/) (2019)
- [Abstractive Text Summarization Using Sequence-to-Sequence RNNs and Beyond](https://mlsanity.com/paper/1602.06023/) (2016)
- [SAMSum Corpus: A Human-annotated Dialogue Dataset for Abstractive Summarization](https://mlsanity.com/paper/1911.12237/) (2019)
- [A Discourse-Aware Attention Model for Abstractive Summarization of Long Documents](https://mlsanity.com/paper/1804.05685/) (2018)
- [QMSum: A New Benchmark for Query-based Multi-domain Meeting Summarization](https://mlsanity.com/paper/2104.05938/) (2021)
- [Efficient Attentions for Long Document Summarization](https://mlsanity.com/paper/2104.02112/) (2021)
- [Recursively Summarizing Books with Human Feedback](https://mlsanity.com/paper/2109.10862/) (2021)
- [Sparse Sinkhorn Attention](https://mlsanity.com/paper/2002.11296/) (2020)
- [BookSum: A Collection of Datasets for Long-form Narrative Summarization](https://mlsanity.com/paper/2105.08209/) (2021)
- [Bottom-Up Abstractive Summarization](https://mlsanity.com/paper/1808.10792/) (2018)
- [MediaSum: A Large-scale Media Interview Dataset for Dialogue Summarization](https://mlsanity.com/paper/2103.06410/) (2021)
- [On Extractive and Abstractive Neural Document Summarization with Transformer Language Models](https://mlsanity.com/paper/1909.03186/) (2019)
- [SummScreen: A Dataset for Abstractive Screenplay Summarization](https://mlsanity.com/paper/2104.07091/) (2021)
- [DialogLM: Pre-trained Model for Long Dialogue Understanding and Summarization](https://mlsanity.com/paper/2109.02492/) (2021)
- [Abstractive Dialogue Summarization with Sentence-Gated Modeling Optimized by Dialogue Acts](https://mlsanity.com/paper/1809.05715/) (2018)
- [ConvoSumm: Conversation Summarization Benchmark and Improved Abstractive Summarization with Argument Mining](https://mlsanity.com/paper/2106.00829/) (2021)
- [Structure-Aware Abstractive Conversation Summarization via Discourse and Action Graphs](https://mlsanity.com/paper/2104.08400/) (2021)
- [DYLE: Dynamic Latent Extraction for Abstractive Long-Input Summarization](https://mlsanity.com/paper/2110.08168/) (2021)
- [Hierarchical Learning for Generation with Long Source Sequences](https://mlsanity.com/paper/2104.07545/) (2021)
- [Dialogue Discourse-Aware Graph Model and Data Augmentation for Meeting Summarization](https://mlsanity.com/paper/2012.03502/) (2020)
- [An Exploratory Study on Long Dialogue Summarization: What Works and What's Next](https://mlsanity.com/paper/2109.04609/) (2021)
- [End-to-End Segmentation-based News Summarization](https://mlsanity.com/paper/2110.07850/) (2021)
- [SEAL: Segment-wise Extractive-Abstractive Long-form Text Summarization](https://mlsanity.com/paper/2006.10213/) (2020)

## Built on by (most cited first, 30 of 65)

- [Chain of Agents: Large Language Models Collaborating on Long-Context Tasks](https://mlsanity.com/paper/2406.02818/) (2024)
- [Unlimiformer: Long-Range Transformers with Unlimited Length Input](https://mlsanity.com/paper/2305.01625/) (2023)
- [SCM: Enhancing Large Language Model with Self-Controlled Memory Framework](https://mlsanity.com/paper/2304.13343/) (2023)
- [MeetingBank: A Benchmark Dataset for Meeting Summarization](https://mlsanity.com/paper/2305.17529/) (2023)
- [Found in the Middle: How Language Models Use Long Contexts Better via Plug-and-Play Positional Encoding](https://mlsanity.com/paper/2403.04797/) (2024)
- [BAMBOO: A Comprehensive Benchmark for Evaluating Long Text Modeling Capacities of Large Language Models](https://mlsanity.com/paper/2309.13345/) (2023)
- [A Survey on Long Text Modeling with Transformers](https://mlsanity.com/paper/2302.14502/) (2023)
- [Exploring Neural Models for Query-Focused Summarization](https://mlsanity.com/paper/2112.07637/) (2021)
- [Prompted Opinion Summarization with GPT-3.5](https://mlsanity.com/paper/2211.15914/) (2022)
- [How Far are We from Robust Long Abstractive Summarization?](https://mlsanity.com/paper/2210.16732/) (2022)
- [Applicability of Large Language Models and Generative Models for Legal Case Judgement Summarization](https://mlsanity.com/paper/2407.12848/) (2024)
- [SNaC: Coherence Error Detection for Narrative Summarization](https://mlsanity.com/paper/2205.09641/) (2022)
- [Incorporating Distributions of Discourse Structure for Long Document Abstractive Summarization](https://mlsanity.com/paper/2305.16784/) (2023)
- [Hierarchical3D Adapters for Long Video-to-text Summarization](https://mlsanity.com/paper/2210.04829/) (2022)
- [Knowledge Planning in Large Language Models for Domain-Aligned Counseling Summarization](https://mlsanity.com/paper/2409.14907/) (2024)
- [Select and Summarize: Scene Saliency for Movie Script Summarization](https://mlsanity.com/paper/2404.03561/) (2024)
- [NexusSum: Hierarchical LLM Agents for Long-Form Narrative Summarization](https://mlsanity.com/paper/2505.24575/) (2025)
- [Leveraging Locality in Abstractive Text Summarization](https://mlsanity.com/paper/2205.12476/) (2022)
- [Abstractive Meeting Summarization: A Survey](https://mlsanity.com/paper/2208.04163/) (2022)
- [Generating EDU Extracts for Plan-Guided Summary Re-Ranking](https://mlsanity.com/paper/2305.17779/) (2023)
- [Chunk, Align, Select: A Simple Long-sequence Processing Method for Transformers](https://mlsanity.com/paper/2308.13191/) (2023)
- [Agent-as-Judge for Factual Summarization of Long Narratives](https://mlsanity.com/paper/2501.09993/) (2025)
- [MovieSum: An Abstractive Summarization Dataset for Movie Screenplays](https://mlsanity.com/paper/2408.06281/) (2024)
- [Factorizing Content and Budget Decisions in Abstractive Summarization of Long Documents](https://mlsanity.com/paper/2205.12486/) (2022)
- [Multi-Party Conversational Agents: A Survey](https://mlsanity.com/paper/2505.18845/) (2025)
- [The Current State of Summarization](https://mlsanity.com/paper/2305.04853/) (2023)
- [Adapter-based Selective Knowledge Distillation for Federated Multi-domain Meeting Summarization](https://mlsanity.com/paper/2308.03275/) (2023)
- [ORCHID: A Chinese Debate Corpus for Target-Independent Stance Detection and Argumentative Dialogue Summarization](https://mlsanity.com/paper/2410.13667/) (2024)
- [Enhancing Dialogue Summarization with Topic-Aware Global- and Local- Level Centrality](https://mlsanity.com/paper/2301.12376/) (2023)
- [Hybrid Long Document Summarization using C2F-FAR and ChatGPT: A Practical Study](https://mlsanity.com/paper/2306.01169/) (2023)

## Abstract (arXiv)

Text summarization helps readers capture salient information from documents, news, interviews, and meetings. However, most state-of-the-art pretrained language models (LM) are unable to efficiently process long text for many summarization tasks. In this paper, we propose SummN, a simple, flexible, and effective multi-stage framework for input texts that are longer than the maximum context length of typical pretrained LMs. SummN first splits the data samples and generates a coarse summary in multiple stages and then produces the final fine-grained summary based on it. Our framework can process input text of arbitrary length by adjusting the number of stages while keeping the LM input size fixed. Moreover, it can deal with both single-source documents and dialogues, and it can be used on top of different backbone abstractive summarization models. To the best of our knowledge, SummN is the first multi-stage split-then-summarize framework for long input summarization. Our experiments demonstrate that SummN outperforms previous state-of-the-art methods by improving ROUGE scores on three long meeting summarization datasets AMI, ICSI, and QMSum, two long TV series datasets from SummScreen, and a long document summarization dataset GovReport. Our data and code are available at https://github.com/psunlpgroup/Summ-N.