Reading Between the Waves: Robust Topic Segmentation Using Inter-Sentence Audio Features
Not on the map. Why?
Abstract (arXiv)
Spoken content, such as online videos and podcasts, often spans multiple topics, which makes automatic topic segmentation essential for user navigation and downstream applications. However, current methods do not fully leverage acoustic features, leaving room for improvement. We propose a multi-modal approach that fine-tunes both a text encoder and a Siamese audio encoder, capturing acoustic cues around sentence boundaries. Experiments on a large-scale dataset of YouTube videos show substantial gains over text-only and multi-modal baselines. Our model also proves more resilient to ASR noise and outperforms a larger text-only baseline on three additional datasets in Portuguese, German, and English, underscoring the value of learned acoustic features for robust topic segmentation.
Key facts. Cited by 1 paper in the map (data through 7 Oct 2026). Builds on 20 of its 42 references that are in the map. First posted on arXiv 6 Feb 2026.
Builds on
20 of 42 references.
References · 20
- 2019RoBERTa: A Robustly Optimized BERT Pretraining Approachcited by 13,266
- 2019Sentence-BERT: Sentence Embeddings using Siamese BERT-Networkscited by 9,049
- 2021RoFormer: Enhanced Transformer with Rotary Position Embeddingcited by 4,581
- 2022Robust Speech Recognition via Large-Scale Weak Supervisioncited by 3,972
- 2020wav2vec 2.0: A Framework for Self-Supervised Learning of Speech Representationscited by 3,853
- 2020MiniLM: Deep Self-Attention Distillation for Task-Agnostic Compression of Pre-Trained Transformerscited by 1,414
- 2020MPNet: Masked and Permuted Pre-training for Language Understandingcited by 973
- 2018Text Segmentation as a Supervised Learning Taskcited by 76
- 2020Text Segmentation by Cross Segment Attentioncited by 54
- 2021UniSpeech-SAT: Universal Speech Representation Learning with Speaker Aware Pre-Trainingcited by 43
- 2021Unsupervised Topic Segmentation of Meetings with BERT Embeddingscited by 30
- 2023Uncovering the Potential of ChatGPT for Discourse Analysis in Dialogue: An Empirical Studycited by 24
8 more
- 2024From Text Segmentation to Smart Chaptering: A Novel Benchmark for Structuring Video Transcriptionscited by 16
- 2022Topic Segmentation in the Wild: Towards Segmentation of Semi-structured & Unstructured Chatscited by 5
- 2020Improving automated segmentation of radio shows with audio embeddingscited by 4
- 2026Towards Multi-Level Transcript Segmentation: LoRA Fine-Tuning for Table-of-Contents Generationcited by 4
- 2022Unsupervised Audio-Visual Lecture Segmentationcited by 3
- 2023A Comparative Study of Pre-trained Speech and Audio Embeddings for Speech Emotion Recognitioncited by 3
- 2023Unsupervised Dialogue Topic Segmentation in Hyperdimensional Spacecited by 3
- 2024Multimodal Fusion and Coherence Modeling for Video Topic Segmentationcited by 1
Built on by
Citing papers per year: 2026: 1.
1 paper.