# Hierarchical Summarization for Longform Spoken Dialog

arXiv: https://arxiv.org/abs/2108.09597 · HTML: https://mlsanity.com/paper/2108.09597/

## Key facts

- Cited by 9 papers in the map (data through 14 Oct 2026).
- Builds on 13 of its 42 references that are in the map.
- First posted on arXiv 21 Aug 2021.

## Builds on (most cited first, 13 of 13)

- [Sentence-BERT: Sentence Embeddings using Siamese BERT-Networks](https://mlsanity.com/paper/1908.10084/) (2019)
- [BERTScore: Evaluating Text Generation with BERT](https://mlsanity.com/paper/1904.09675/) (2019)
- [HuggingFace's Transformers: State-of-the-art Natural Language Processing](https://mlsanity.com/paper/1910.03771/) (2019)
- [PEGASUS: Pre-training with Extracted Gap-sentences for Abstractive Summarization](https://mlsanity.com/paper/1912.08777/) (2019)
- [On Faithfulness and Factuality in Abstractive Summarization](https://mlsanity.com/paper/2005.00661/) (2020)
- [SAMSum Corpus: A Human-annotated Dialogue Dataset for Abstractive Summarization](https://mlsanity.com/paper/1911.12237/) (2019)
- [AllenNLP: A Deep Semantic Natural Language Processing Platform](https://mlsanity.com/paper/1803.07640/) (2018)
- [Discourse-Aware Neural Extractive Text Summarization](https://mlsanity.com/paper/1910.14142/) (2019)
- [Unsupervised Abstractive Meeting Summarization with Multi-Sentence Compression and Budgeted Submodular Maximization](https://mlsanity.com/paper/1805.05271/) (2018)
- [Reducing Quantity Hallucinations in Abstractive Summarization](https://mlsanity.com/paper/2009.13312/) (2020)
- [A Comparison of Online Automatic Speech Recognition Systems and the Nonverbal Responses to Unintelligible Speech](https://mlsanity.com/paper/1904.12403/) (2019)
- [A Two-Phase Approach for Abstractive Podcast Summarization](https://mlsanity.com/paper/2011.08291/) (2020)
- [PodSumm -- Podcast Audio Summarization](https://mlsanity.com/paper/2009.10315/) (2020)

## Built on by (most cited first, 9 of 9)

- [Beyond Text Generation: Supporting Writers with Continuous Automatic Text Summaries](https://mlsanity.com/paper/2208.09323/) (2022)
- [CONFIT: Toward Faithful Dialogue Summarization with Linguistically-Informed Contrastive Fine-tuning](https://mlsanity.com/paper/2112.08713/) (2021)
- [Paper Plain: Making Medical Research Papers Approachable to Healthcare Consumers with Natural Language Processing](https://mlsanity.com/paper/2203.00130/) (2022)
- [Lotus: Creating Short Videos From Long Videos With Abstractive and Extractive Summarization](https://mlsanity.com/paper/2502.07096/) (2025)
- [Towards Multi-Level Transcript Segmentation: LoRA Fine-Tuning for Table-of-Contents Generation](https://mlsanity.com/paper/2601.02128/) (2026)
- [Taxonomy of Abstractive Dialogue Summarization: Scenarios, Approaches and Future Directions](https://mlsanity.com/paper/2210.09894/) (2022)
- [Supporting Experts with a Multimodal Machine-Learning-Based Tool for Human Behavior Analysis of Conversational Videos](https://mlsanity.com/paper/2402.11145/) (2024)
- [Breaking the Curse of Knowledge: Designing Personalized Jargon Support for Real-Time Online Meetings](https://mlsanity.com/paper/2508.10239/) (2025)
- [Don't Scroll Back: Missing-Evidence Memory for Streaming Dialogue Summarization](https://mlsanity.com/paper/2608.09043/) (2026)

## Abstract (arXiv)

Every day we are surrounded by spoken dialog. This medium delivers rich diverse streams of information auditorily; however, systematically understanding dialog can often be non-trivial. Despite the pervasiveness of spoken dialog, automated speech understanding and quality information extraction remains markedly poor, especially when compared to written prose. Furthermore, compared to understanding text, auditory communication poses many additional challenges such as speaker disfluencies, informal prose styles, and lack of structure. These concerns all demonstrate the need for a distinctly speech tailored interactive system to help users understand and navigate the spoken language domain. While individual automatic speech recognition (ASR) and text summarization methods already exist, they are imperfect technologies; neither consider user purpose and intent nor address spoken language induced complications. Consequently, we design a two stage ASR and text summarization pipeline and propose a set of semantic segmentation and merging algorithms to resolve these speech modeling challenges. Our system enables users to easily browse and navigate content as well as recover from errors in these underlying technologies. Finally, we present an evaluation of the system which highlights user preference for hierarchical summarization as a tool to quickly skim audio and identify content of interest to the user.