# Sequence Model with Self-Adaptive Sliding Window for Efficient Spoken Document Segmentation

arXiv: https://arxiv.org/abs/2107.09278 · HTML: https://mlsanity.com/paper/2107.09278/

## Key facts

- Cited by 14 papers in the map (data through 14 Oct 2026).
- Builds on 9 of its 20 references that are in the map.
- First posted on arXiv 20 Jul 2021.

## Builds on (most cited first, 9 of 9)

- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [RoBERTa: A Robustly Optimized BERT Pretraining Approach](https://mlsanity.com/paper/1907.11692/) (2019)
- [Google's Neural Machine Translation System: Bridging the Gap between Human and Machine Translation](https://mlsanity.com/paper/1609.08144/) (2016)
- [Text Segmentation as a Supervised Learning Task](https://mlsanity.com/paper/1803.09337/) (2018)
- [Text Segmentation by Cross Segment Attention](https://mlsanity.com/paper/2004.14535/) (2020)
- [Pretraining with Contrastive Sentence Objectives Improves Discourse Performance of Language Models](https://mlsanity.com/paper/2005.10389/) (2020)
- [Two-Level Transformer and Auxiliary Coherence Modeling for Improved Text Segmentation](https://mlsanity.com/paper/2001.00891/) (2020)
- [Attention-based Neural Text Segmentation](https://mlsanity.com/paper/1808.09935/) (2018)

## Built on by (most cited first, 14 of 14)

- [A Survey on Knowledge-Oriented Retrieval-Augmented Generation](https://mlsanity.com/paper/2503.10677/) (2025)
- [MoC: Mixtures of Text Chunking Learners for Retrieval-Augmented Generation System](https://mlsanity.com/paper/2503.09600/) (2025)
- [Meta-Chunking: Learning Text Segmentation and Semantic Completion via Logical Perception](https://mlsanity.com/paper/2410.12788/) (2024)
- [Improving Long Document Topic Segmentation Models With Enhanced Coherence Modeling](https://mlsanity.com/paper/2310.11772/) (2023)
- [HiChunk: Evaluating and Enhancing Retrieval-Augmented Generation with Hierarchical Chunking](https://mlsanity.com/paper/2509.11552/) (2025)
- [PODTILE: Facilitating Podcast Episode Browsing with Auto-generated Chapters](https://mlsanity.com/paper/2410.16148/) (2024)
- [Recent Trends in Linear Text Segmentation: a Survey](https://mlsanity.com/paper/2411.16613/) (2024)
- [Document Summarization with Text Segmentation](https://mlsanity.com/paper/2301.08817/) (2023)
- [MUG: A General Meeting Understanding and Generation Benchmark](https://mlsanity.com/paper/2303.13939/) (2023)
- [Recording for Eyes, Not Echoing to Ears: Contextualized Spoken-to-Written Conversion of ASR Transcripts](https://mlsanity.com/paper/2408.09688/) (2024)
- [CrossFormer: Cross-Segment Semantic Fusion for Document Segmentation](https://mlsanity.com/paper/2503.23671/) (2025)
- [Multimodal Fusion and Coherence Modeling for Video Topic Segmentation](https://mlsanity.com/paper/2408.00365/) (2024)
- [FEAT: A Multi-Agent Forensic AI System with Domain-Adapted Large Language Model for Automated Cause-of-Death Analysis](https://mlsanity.com/paper/2508.07950/) (2025)
- [QChunker: Learning Question-Aware Text Chunking for Domain RAG via Multi-Agent Debate](https://mlsanity.com/paper/2603.11650/) (2026)

## Abstract (arXiv)

Transcripts generated by automatic speech recognition (ASR) systems for spoken documents lack structural annotations such as paragraphs, significantly reducing their readability. Automatically predicting paragraph segmentation for spoken documents may both improve readability and downstream NLP performance such as summarization and machine reading comprehension. We propose a sequence model with self-adaptive sliding window for accurate and efficient paragraph segmentation. We also propose an approach to exploit phonetic information, which significantly improves robustness of spoken document segmentation to ASR errors. Evaluations are conducted on the English Wiki-727K document segmentation benchmark, a Chinese Wikipedia-based document segmentation dataset we created, and an in-house Chinese spoken document dataset. Our proposed model outperforms the state-of-the-art (SOTA) model based on the same BERT-Base, increasing segmentation F1 on the English benchmark by 4.2 points and on Chinese datasets by 4.3-10.1 points, while reducing inference time to less than 1/6 of inference time of the current SOTA.