# An Exploratory Study on Long Dialogue Summarization: What Works and What's Next

arXiv: https://arxiv.org/abs/2109.04609 · HTML: https://mlsanity.com/paper/2109.04609/

## Key facts

- Cited by 28 papers in the map (data through 14 Oct 2026).
- Builds on 20 of its 27 references that are in the map.
- First posted on arXiv 10 Sep 2021.

## Builds on (most cited first, 20 of 20)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [BART: Denoising Sequence-to-Sequence Pre-training for Natural Language Generation, Translation, and Comprehension](https://mlsanity.com/paper/1910.13461/) (2019)
- [Longformer: The Long-Document Transformer](https://mlsanity.com/paper/2004.05150/) (2020)
- [Big Bird: Transformers for Longer Sequences](https://mlsanity.com/paper/2007.14062/) (2020)
- [Don't Give Me the Details, Just the Summary! Topic-Aware Convolutional Neural Networks for Extreme Summarization](https://mlsanity.com/paper/1808.08745/) (2018)
- [Teaching Machines to Read and Comprehend](https://mlsanity.com/paper/1506.03340/) (2015)
- [SAMSum Corpus: A Human-annotated Dialogue Dataset for Abstractive Summarization](https://mlsanity.com/paper/1911.12237/) (2019)
- [A Discourse-Aware Attention Model for Abstractive Summarization of Long Documents](https://mlsanity.com/paper/1804.05685/) (2018)
- [Multi-News: a Large-Scale Multi-Document Summarization Dataset and Abstractive Hierarchical Model](https://mlsanity.com/paper/1906.01749/) (2019)
- [QMSum: A New Benchmark for Query-based Multi-domain Meeting Summarization](https://mlsanity.com/paper/2104.05938/) (2021)
- [BIGPATENT: A Large-Scale Dataset for Abstractive and Coherent Summarization](https://mlsanity.com/paper/1906.03741/) (2019)
- [BillSum: A Corpus for Automatic Summarization of US Legislation](https://mlsanity.com/paper/1910.00523/) (2019)
- [MediaSum: A Large-scale Media Interview Dataset for Dialogue Summarization](https://mlsanity.com/paper/2103.06410/) (2021)
- [SummScreen: A Dataset for Abstractive Screenplay Summarization](https://mlsanity.com/paper/2104.07091/) (2021)
- [Multi-View Sequence-to-Sequence Models with Conversational Structure for Abstractive Dialogue Summarization](https://mlsanity.com/paper/2010.01672/) (2020)
- [Unsupervised Abstractive Meeting Summarization with Multi-Sentence Compression and Budgeted Submodular Maximization](https://mlsanity.com/paper/1805.05271/) (2018)
- [Diversity driven Attention Model for Query-based Abstractive Summarization](https://mlsanity.com/paper/1704.08300/) (2017)
- [Structure-Aware Abstractive Conversation Summarization via Discourse and Action Graphs](https://mlsanity.com/paper/2104.08400/) (2021)
- [Hierarchical Learning for Generation with Long Source Sequences](https://mlsanity.com/paper/2104.07545/) (2021)

## Built on by (most cited first, 28 of 28)

- [News Summarization and Evaluation in the Era of GPT-3](https://mlsanity.com/paper/2209.12356/) (2022)
- [Chain of Agents: Large Language Models Collaborating on Long-Context Tasks](https://mlsanity.com/paper/2406.02818/) (2024)
- [An Empirical Survey on Long Document Summarization: Datasets, Models and Metrics](https://mlsanity.com/paper/2207.00939/) (2022)
- [Summ^N: A Multi-Stage Summarization Framework for Long Input Dialogues and Documents](https://mlsanity.com/paper/2110.10150/) (2021)
- [Exploring Neural Models for Query-Focused Summarization](https://mlsanity.com/paper/2112.07637/) (2021)
- [DYLE: Dynamic Latent Extraction for Abstractive Long-Input Summarization](https://mlsanity.com/paper/2110.08168/) (2021)
- [How Far are We from Robust Long Abstractive Summarization?](https://mlsanity.com/paper/2210.16732/) (2022)
- [Hierarchical3D Adapters for Long Video-to-text Summarization](https://mlsanity.com/paper/2210.04829/) (2022)
- [Summaries, Highlights, and Action items: Design, implementation and evaluation of an LLM-powered meeting recap system](https://mlsanity.com/paper/2307.15793/) (2023)
- [Leveraging Large Text Corpora for End-to-End Speech Summarization](https://mlsanity.com/paper/2303.00978/) (2023)
- [What's under the hood: Investigating Automatic Metrics on Meeting Summarization](https://mlsanity.com/paper/2404.11124/) (2024)
- [An End-to-End Dialogue Summarization System for Sales Calls](https://mlsanity.com/paper/2204.12951/) (2022)
- [Multi-Party Conversational Agents: A Survey](https://mlsanity.com/paper/2505.18845/) (2025)
- [Adapter-based Selective Knowledge Distillation for Federated Multi-domain Meeting Summarization](https://mlsanity.com/paper/2308.03275/) (2023)
- [Prompting Large Language Models with Audio for General-Purpose Speech Summarization](https://mlsanity.com/paper/2406.05968/) (2024)
- [Enhancing Dialogue Summarization with Topic-Aware Global- and Local- Level Centrality](https://mlsanity.com/paper/2301.12376/) (2023)
- [Summarizing Speech: A Comprehensive Survey](https://mlsanity.com/paper/2504.08024/) (2025)
- [Effectiveness of French Language Models on Abstractive Dialogue Summarization Task](https://mlsanity.com/paper/2207.08305/) (2022)
- [Taxonomy of Abstractive Dialogue Summarization: Scenarios, Approaches and Future Directions](https://mlsanity.com/paper/2210.09894/) (2022)
- [Meeting Summarization: A Survey of the State of the Art](https://mlsanity.com/paper/2212.08206/) (2022)
- [From Transcripts to Insights: Uncovering Corporate Risks Using Generative AI](https://mlsanity.com/paper/2310.17721/) (2023)
- [Aspect-based Meeting Transcript Summarization: A Two-Stage Approach with Weak Supervision on Sentence Classification](https://mlsanity.com/paper/2311.04292/) (2023)
- [Can one size fit all?: Measuring Failure in Multi-Document Summarization Domain Transfer](https://mlsanity.com/paper/2503.15768/) (2025)
- [LSTM-MAS: A Long Short-Term Memory Inspired Multi-Agent System for Long-Context Understanding](https://mlsanity.com/paper/2601.11913/) (2026)
- [Meeting Summarization with Pre-training and Clustering Methods](https://mlsanity.com/paper/2111.08210/) (2021)
- [Towards End-to-end Speech-to-text Summarization](https://mlsanity.com/paper/2306.05432/) (2023)
- [DIAL-SUMMER: A Structured Evaluation Framework of Hierarchical Errors in Dialogue Summaries](https://mlsanity.com/paper/2602.08149/) (2026)
- [Don't Scroll Back: Missing-Evidence Memory for Streaming Dialogue Summarization](https://mlsanity.com/paper/2608.09043/) (2026)

## Abstract (arXiv)

Dialogue summarization helps readers capture salient information from long conversations in meetings, interviews, and TV series. However, real-world dialogues pose a great challenge to current summarization models, as the dialogue length typically exceeds the input limits imposed by recent transformer-based pre-trained models, and the interactive nature of dialogues makes relevant information more context-dependent and sparsely distributed than news articles. In this work, we perform a comprehensive study on long dialogue summarization by investigating three strategies to deal with the lengthy input problem and locate relevant information: (1) extended transformer models such as Longformer, (2) retrieve-then-summarize pipeline models with several dialogue utterance retrieval methods, and (3) hierarchical dialogue encoding models such as HMNet. Our experimental results on three long dialogue datasets (QMSum, MediaSum, SummScreen) show that the retrieve-then-summarize pipeline models yield the best performance. We also demonstrate that the summary quality can be further improved with a stronger retrieval model and pretraining on proper external summarization datasets.