# Self-Distillation Prototypes Network: Learning Robust Speaker Representations without Supervision

arXiv: https://arxiv.org/abs/2406.11169 · HTML: https://mlsanity.com/paper/2406.11169/

## Key facts

- Cited by 5 papers in the map (data through 14 Oct 2026).
- Placed in Speech and audio distillation, one of 12 directions in Knowledge distillation.
- Builds on 23 of its 35 references that are in the map.
- First posted on arXiv 17 Jun 2024.

## Placed in

- Knowledge distillation / [Speech and audio distillation](https://mlsanity.com/distill/speech-and-audio-distillation/)

## Builds on (most cited first, 23 of 23)

- [A Simple Framework for Contrastive Learning of Visual Representations](https://mlsanity.com/paper/2002.05709/) (2020)
- [Representation Learning with Contrastive Predictive Coding](https://mlsanity.com/paper/1807.03748/) (2018)
- [Momentum Contrast for Unsupervised Visual Representation Learning](https://mlsanity.com/paper/1911.05722/) (2019)
- [Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shift](https://mlsanity.com/paper/1502.03167/) (2015)
- [Emerging Properties in Self-Supervised Vision Transformers](https://mlsanity.com/paper/2104.14294/) (2021)
- [Bootstrap your own latent: A new approach to self-supervised Learning](https://mlsanity.com/paper/2006.07733/) (2020)
- [Sinkhorn Distances: Lightspeed Computation of Optimal Transportation Distances](https://mlsanity.com/paper/1306.0895/) (2013)
- [SpecAugment: A Simple Data Augmentation Method for Automatic Speech Recognition](https://mlsanity.com/paper/1904.08779/) (2019)
- [VoxCeleb2: Deep Speaker Recognition](https://mlsanity.com/paper/1806.05622/) (2018)
- [VoxCeleb: a large-scale speaker identification dataset](https://mlsanity.com/paper/1706.08612/) (2017)
- [ECAPA-TDNN: Emphasized Channel Attention, Propagation and Aggregation in TDNN Based Speaker Verification](https://mlsanity.com/paper/2005.07143/) (2020)
- [Self-Normalizing Neural Networks](https://mlsanity.com/paper/1706.02515/) (2017)
- [MUSAN: A Music, Speech, and Noise Corpus](https://mlsanity.com/paper/1510.08484/) (2015)
- [Self-supervised Text-independent Speaker Verification using Prototypical Momentum Contrastive Learning](https://mlsanity.com/paper/2012.07178/) (2020)
- [Self-Supervised Speaker Verification with Simple Siamese Network and Self-Supervised Regularization](https://mlsanity.com/paper/2112.04459/) (2021)
- [Pushing the limits of self-supervised speaker verification using regularized distillation framework](https://mlsanity.com/paper/2211.04168/) (2022)
- [Self-supervised Speaker Recognition with Loss-gated Learning](https://mlsanity.com/paper/2110.03869/) (2021)
- [Disentangling Voice and Content with Self-Supervision for Speaker Recognition](https://mlsanity.com/paper/2310.01128/) (2023)
- [C3-DINO: Joint Contrastive and Non-contrastive Self-Supervised Learning for Speaker Verification](https://mlsanity.com/paper/2208.07446/) (2022)
- [Self-Supervised Learning with Cluster-Aware-DINO for High-Performance Robust Speaker Verification](https://mlsanity.com/paper/2304.05754/) (2023)
- [A comprehensive study on self-supervised distillation for speaker representation learning](https://mlsanity.com/paper/2210.15936/) (2022)
- [Contrastive-mixup learning for improved speaker verification](https://mlsanity.com/paper/2202.10672/) (2022)
- [Curriculum learning for self-supervised speaker verification](https://mlsanity.com/paper/2203.14525/) (2022)

## Built on by (most cited first, 5 of 5)

- [Pushing the Frontiers of Self-Distillation Prototypes Network with Dimension Regularization and Score Normalization](https://mlsanity.com/paper/2505.13826/) (2025)
- [VoxAging: Continuously Tracking Speaker Aging with a Large-Scale Longitudinal Dataset in English and Mandarin](https://mlsanity.com/paper/2505.21445/) (2025)
- [VoxKnesset: A Large-Scale Longitudinal Hebrew Speech Dataset for Aging Speaker Modeling](https://mlsanity.com/paper/2603.01270/) (2026)
- [Self-Supervised Learning for Speaker Recognition: A study and review](https://mlsanity.com/paper/2602.10829/) (2026)
- [Do Speech Tokens Leak Voiceprints? Speaker Inversion Attacks Against End-to-End Speech Language Models](https://mlsanity.com/paper/2607.16870/) (2026)

## Abstract (arXiv)

Training speaker-discriminative and robust speaker verification systems without explicit speaker labels remains a persisting challenge. In this paper, we propose a new self-supervised speaker verification approach, Self-Distillation Prototypes Network (SDPN), which effectively facilitates self-supervised speaker representation learning. SDPN assigns the representation of the augmented views of an utterance to the same prototypes as the representation of the original view, thereby enabling effective knowledge transfer between the views. Originally, due to the lack of negative pairs in the SDPN training process, the network tends to align positive pairs very closely in the embedding space, a phenomenon known as model collapse. To alleviate this problem, we introduce a diversity regularization term to embeddings in SDPN. Comprehensive experiments on the VoxCeleb datasets demonstrate the superiority of SDPN in self-supervised speaker verification. SDPN sets a new state-of-the-art on the VoxCeleb1 speaker verification evaluation benchmark, achieving Equal Error Rate 1.80%, 1.99%, and 3.62% for trial VoxCeleb1-O, VoxCeleb1-E and VoxCeleb1-H respectively, without using any speaker labels in training.