# A comprehensive study on self-supervised distillation for speaker representation learning

arXiv: https://arxiv.org/abs/2210.15936 · HTML: https://mlsanity.com/paper/2210.15936/

## Key facts

- Cited by 8 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 28 Oct 2022.

## Built on by (most cited first, 8 of 8)

- [What Do Self-Supervised Speech and Speaker Models Learn? New Findings From a Cross Model Layer-Wise Analysis](https://mlsanity.com/paper/2401.17632/) (2024)
- [Textless Acoustic Model with Self-Supervised Distillation for Noise-Robust Expressive Speech-to-Speech Translation](https://mlsanity.com/paper/2406.02733/) (2024)
- [Self-Distillation Prototypes Network: Learning Robust Speaker Representations without Supervision](https://mlsanity.com/paper/2406.11169/) (2024)
- [A Genealogy of Foundation Models in Remote Sensing](https://mlsanity.com/paper/2504.17177/) (2025)
- [Self-Supervised Frameworks for Speaker Verification via Bootstrapped Positive Sampling](https://mlsanity.com/paper/2501.17772/) (2025)
- [Investigation of Speaker Representation for Target-Speaker Speech Processing](https://mlsanity.com/paper/2410.11243/) (2024)
- [Self-Distillation Prototypes Network: Learning Robust Speaker Representations without Supervision](https://mlsanity.com/paper/2308.02774/) (2023)
- [Self-Supervised Learning for Speaker Recognition: A study and review](https://mlsanity.com/paper/2602.10829/) (2026)

## Abstract (arXiv)

In real application scenarios, it is often challenging to obtain a large amount of labeled data for speaker representation learning due to speaker privacy concerns. Self-supervised learning with no labels has become a more and more promising way to solve it. Compared with contrastive learning, self-distilled approaches use only positive samples in the loss function and thus are more attractive. In this paper, we present a comprehensive study on self-distilled self-supervised speaker representation learning, especially on critical data augmentation. Our proposed strategy of audio perturbation augmentation has pushed the performance of the speaker representation to a new limit. The experimental results show that our model can achieve a new SoTA on Voxceleb1 speaker verification evaluation benchmark ( i.e., equal error rate (EER) 2.505%, 2.473%, and 4.791% for trial Vox1-O, Vox1-E and Vox1-H , respectively), discarding any speaker labels in the training phase.