# Multitask Detection of Speaker Changes, Overlapping Speech and Voice Activity Using wav2vec 2.0

arXiv: https://arxiv.org/abs/2210.14755 · HTML: https://mlsanity.com/paper/2210.14755/

## Key facts

- Cited by 4 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 26 Oct 2022.

## Built on by (most cited first, 4 of 4)

- [USM-SCD: Multilingual Speaker Change Detection Based on Large Pretrained Foundation Models](https://mlsanity.com/paper/2309.08023/) (2023)
- [Unsupervised speech intelligibility assessment with utterance level alignment distance between teacher and learner Wav2Vec-2.0 representations](https://mlsanity.com/paper/2306.08845/) (2023)
- [Towards Robust Overlapping Speech Detection: A Speaker-Aware Progressive Approach Using WavLM](https://mlsanity.com/paper/2505.23207/) (2025)
- [Attention Is Not Always the Answer: Optimizing Voice Activity Detection with Simple Feature Fusion](https://mlsanity.com/paper/2506.01365/) (2025)

## Abstract (arXiv)

Self-supervised learning approaches have lately achieved great success on a broad spectrum of machine learning problems. In the field of speech processing, one of the most successful recent self-supervised models is wav2vec 2.0. In this paper, we explore the effectiveness of this model on three basic speech classification tasks: speaker change detection, overlapped speech detection, and voice activity detection. First, we concentrate on only one task -- speaker change detection -- where our proposed system surpasses the previously reported results on four different corpora, and achieves comparable performance even when trained on out-of-domain data from an artificially designed dataset. Then we expand our approach to tackle all three tasks in a single multitask system with state-of-the-art performance on the AMI corpus. The implementation of the algorithms in this paper is publicly available at https://github.com/mkunes/w2v2_audioFrameClassification.