Towards Robust Overlapping Speech Detection: A Speaker-Aware Progressive Approach Using WavLM
Not on the map. Why?
Abstract (arXiv)
Overlapping Speech Detection (OSD) aims to identify regions where multiple speakers overlap in a conversation, a critical challenge in multi-party speech processing. This work proposes a speaker-aware progressive OSD model that leverages a progressive training strategy to enhance the correlation between subtasks such as voice activity detection (VAD) and overlap detection. To improve acoustic representation, we explore the effectiveness of state-of-the-art self-supervised learning (SSL) models, including WavLM and wav2vec 2.0, while incorporating a speaker attention module to enrich features with frame-level speaker information. Experimental results show that the proposed method achieves state-of-the-art performance, with an F1 score of 82.76% on the AMI test set, demonstrating its robustness and effectiveness in OSD.
Key facts. Cited by 1 paper in the map (data through 14 Oct 2026). Builds on 18 of its 27 references that are in the map. First posted on arXiv 29 May 2025.
Builds on
18 of 27 references.
References · 18
- 2021WavLM: Large-Scale Self-Supervised Pre-Training for Full Stack Speech Processingcited by 1,657
- 2017A Survey on Multi-Task Learningcited by 840
- 2023CAM++: A Fast and Efficient Network for Speaker Verification Using Context-Aware Maskingcited by 102
- 2021End-to-end speaker segmentation for overlap-aware resegmentationcited by 80
- 2021M2MeT: The ICASSP 2022 Multi-Channel Multi-Party Meeting Transcription Challengecited by 71
- 2020Target-Speaker Voice Activity Detection: a Novel Approach for Multi-Speaker Diarization in a Dinner Party Scenariocited by 68
- 2021ECAPA-TDNN Embeddings for Speaker Diarizationcited by 47
- 2019Overlap-aware diarization: resegmentation using neural end-to-end overlapped speech detectioncited by 27
- 2024A robust audio deepfake detection system via multi-view featurecited by 19
- 2021Three-class Overlapped Speech Detection using a Convolutional Recurrent Neural Networkcited by 8
- 2022Multitask Detection of Speaker Changes, Overlapping Speech and Voice Activity Using wav2vec 2.0cited by 4
- 2023Frame-wise and overlap-robust speaker embeddings for meeting diarizationcited by 4
6 more
- 2024LibriheavyMix: A 20,000-Hour Dataset for Single-Channel Reverberant Multi-Talker Speech Separation, ASR and Speaker Diarizationcited by 4
- 2021BeamTransformer: Microphone Array-based Overlapping Speech Detectioncited by 2
- 2023Joint speech and overlap detection: a benchmark over multiple audio setup and speech domainscited by 2
- 2024Unified Audio Event Detectioncited by 2
- 2022Spatial-aware Speaker Diarization for Multi-channel Multi-party Meetingcited by 1
- 2023Large-Scale Learning on Overlapped Speech Detection: New Benchmark and New General Systemcited by 1
Built on by
Citing papers per year: 2026: 1.
1 paper.