# SA-WavLM: Speaker-Aware Self-Supervised Pre-training for Mixture Speech

arXiv: https://arxiv.org/abs/2407.02826 · HTML: https://mlsanity.com/paper/2407.02826/

## Key facts

- Cited by 4 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 3 Jul 2024.

## Built on by (most cited first, 4 of 4)

- [Speech Foundation Model Ensembles for the Controlled Singing Voice Deepfake Detection (CtrSVDD) Challenge 2024](https://mlsanity.com/paper/2409.02302/) (2024)
- [VocalAgent: Large Language Models for Vocal Health Diagnostics with Safety-Aware Evaluation](https://mlsanity.com/paper/2505.13577/) (2025)
- [SNAP: Speaker Nulling for Artifact Projection in Speech Deepfake Detection](https://mlsanity.com/paper/2603.20686/) (2026)
- [SepRQ : Self-Supervised Speech Mixture Representation Learning via Mask-Free, Multi-Scale Source Separation](https://mlsanity.com/paper/2610.04690/) (2026)

## Abstract (arXiv)

It was shown that pre-trained models with self-supervised learning (SSL) techniques are effective in various downstream speech tasks. However, most such models are trained on single-speaker speech data, limiting their effectiveness in mixture speech. This motivates us to explore pre-training on mixture speech. This work presents SA-WavLM, a novel pre-trained model for mixture speech. Specifically, SA-WavLM follows an "extract-merge-predict" pipeline in which the representations of each speaker in the input mixture are first extracted individually and then merged before the final prediction. In this pipeline, SA-WavLM performs speaker-informed extractions with the consideration of the interactions between different speakers. Furthermore, a speaker shuffling strategy is proposed to enhance the robustness towards the speaker absence. Experiments show that SA-WavLM either matches or improves upon the state-of-the-art pre-trained models.