# Towards Robust Overlapping Speech Detection: A Speaker-Aware Progressive Approach Using WavLM

arXiv: https://arxiv.org/abs/2505.23207 · HTML: https://mlsanity.com/paper/2505.23207/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Builds on 18 of its 27 references that are in the map.
- First posted on arXiv 29 May 2025.

## Builds on (most cited first, 18 of 18)

- [WavLM: Large-Scale Self-Supervised Pre-Training for Full Stack Speech Processing](https://mlsanity.com/paper/2110.13900/) (2021)
- [A Survey on Multi-Task Learning](https://mlsanity.com/paper/1707.08114/) (2017)
- [CAM++: A Fast and Efficient Network for Speaker Verification Using Context-Aware Masking](https://mlsanity.com/paper/2303.00332/) (2023)
- [End-to-end speaker segmentation for overlap-aware resegmentation](https://mlsanity.com/paper/2104.04045/) (2021)
- [M2MeT: The ICASSP 2022 Multi-Channel Multi-Party Meeting Transcription Challenge](https://mlsanity.com/paper/2110.07393/) (2021)
- [Target-Speaker Voice Activity Detection: a Novel Approach for Multi-Speaker Diarization in a Dinner Party Scenario](https://mlsanity.com/paper/2005.07272/) (2020)
- [ECAPA-TDNN Embeddings for Speaker Diarization](https://mlsanity.com/paper/2104.01466/) (2021)
- [Overlap-aware diarization: resegmentation using neural end-to-end overlapped speech detection](https://mlsanity.com/paper/1910.11646/) (2019)
- [A robust audio deepfake detection system via multi-view feature](https://mlsanity.com/paper/2403.01960/) (2024)
- [Three-class Overlapped Speech Detection using a Convolutional Recurrent Neural Network](https://mlsanity.com/paper/2104.02878/) (2021)
- [Multitask Detection of Speaker Changes, Overlapping Speech and Voice Activity Using wav2vec 2.0](https://mlsanity.com/paper/2210.14755/) (2022)
- [Frame-wise and overlap-robust speaker embeddings for meeting diarization](https://mlsanity.com/paper/2306.00625/) (2023)
- [LibriheavyMix: A 20,000-Hour Dataset for Single-Channel Reverberant Multi-Talker Speech Separation, ASR and Speaker Diarization](https://mlsanity.com/paper/2409.00819/) (2024)
- [BeamTransformer: Microphone Array-based Overlapping Speech Detection](https://mlsanity.com/paper/2109.04049/) (2021)
- [Joint speech and overlap detection: a benchmark over multiple audio setup and speech domains](https://mlsanity.com/paper/2307.13012/) (2023)
- [Unified Audio Event Detection](https://mlsanity.com/paper/2409.08552/) (2024)
- [Spatial-aware Speaker Diarization for Multi-channel Multi-party Meeting](https://mlsanity.com/paper/2209.12002/) (2022)
- [Large-Scale Learning on Overlapped Speech Detection: New Benchmark and New General System](https://mlsanity.com/paper/2308.05987/) (2023)

## Built on by (most cited first, 1 of 1)

- [When Misinformation Speaks and Converses: Rethinking Fact-Checking in Audio Platforms](https://mlsanity.com/paper/2604.16767/) (2026)

## Abstract (arXiv)

Overlapping Speech Detection (OSD) aims to identify regions where multiple speakers overlap in a conversation, a critical challenge in multi-party speech processing. This work proposes a speaker-aware progressive OSD model that leverages a progressive training strategy to enhance the correlation between subtasks such as voice activity detection (VAD) and overlap detection. To improve acoustic representation, we explore the effectiveness of state-of-the-art self-supervised learning (SSL) models, including WavLM and wav2vec 2.0, while incorporating a speaker attention module to enrich features with frame-level speaker information. Experimental results show that the proposed method achieves state-of-the-art performance, with an F1 score of 82.76% on the AMI test set, demonstrating its robustness and effectiveness in OSD.