# Pushing the Frontiers of Self-Distillation Prototypes Network with Dimension Regularization and Score Normalization

arXiv: https://arxiv.org/abs/2505.13826 · HTML: https://mlsanity.com/paper/2505.13826/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Placed in Speech and audio distillation, one of 12 directions in Knowledge distillation.
- Builds on 16 of its 24 references that are in the map.
- First posted on arXiv 20 May 2025.

## Placed in

- Knowledge distillation / [Speech and audio distillation](https://mlsanity.com/distill/speech-and-audio-distillation/)

## Builds on (most cited first, 16 of 16)

- [A Simple Framework for Contrastive Learning of Visual Representations](https://mlsanity.com/paper/2002.05709/) (2020)
- [Emerging Properties in Self-Supervised Vision Transformers](https://mlsanity.com/paper/2104.14294/) (2021)
- [Bootstrap your own latent: A new approach to self-supervised Learning](https://mlsanity.com/paper/2006.07733/) (2020)
- [Barlow Twins: Self-Supervised Learning via Redundancy Reduction](https://mlsanity.com/paper/2103.03230/) (2021)
- [VICReg: Variance-Invariance-Covariance Regularization for Self-Supervised Learning](https://mlsanity.com/paper/2105.04906/) (2021)
- [VoxCeleb2: Deep Speaker Recognition](https://mlsanity.com/paper/1806.05622/) (2018)
- [VoxCeleb: a large-scale speaker identification dataset](https://mlsanity.com/paper/1706.08612/) (2017)
- [On the duality between contrastive and non-contrastive self-supervised learning](https://mlsanity.com/paper/2206.02574/) (2022)
- [Self-supervised Text-independent Speaker Verification using Prototypical Momentum Contrastive Learning](https://mlsanity.com/paper/2012.07178/) (2020)
- [Self-Supervised Speaker Verification with Simple Siamese Network and Self-Supervised Regularization](https://mlsanity.com/paper/2112.04459/) (2021)
- [Pushing the limits of self-supervised speaker verification using regularized distillation framework](https://mlsanity.com/paper/2211.04168/) (2022)
- [Disentangling Voice and Content with Self-Supervision for Speaker Recognition](https://mlsanity.com/paper/2310.01128/) (2023)
- [C3-DINO: Joint Contrastive and Non-contrastive Self-Supervised Learning for Speaker Verification](https://mlsanity.com/paper/2208.07446/) (2022)
- [Contrastive-mixup learning for improved speaker verification](https://mlsanity.com/paper/2202.10672/) (2022)
- [Curriculum learning for self-supervised speaker verification](https://mlsanity.com/paper/2203.14525/) (2022)
- [Self-Distillation Prototypes Network: Learning Robust Speaker Representations without Supervision](https://mlsanity.com/paper/2406.11169/) (2024)

## Built on by (most cited first, 1 of 1)

- [Interpolating Speaker Identities in Embedding Space for Data Expansion](https://mlsanity.com/paper/2508.19210/) (2025)

## Abstract (arXiv)

Developing robust speaker verification (SV) systems without speaker labels has been a longstanding challenge. Earlier research has highlighted a considerable performance gap between self-supervised and fully supervised approaches. In this paper, we enhance the non-contrastive self-supervised framework, Self-Distillation Prototypes Network (SDPN), by introducing dimension regularization that explicitly addresses the collapse problem through the application of regularization terms to speaker embeddings. Moreover, we integrate score normalization techniques from fully supervised SV to further bridge the gap toward supervised verification performance. SDPN with dimension regularization and score normalization sets a new state-of-the-art on the VoxCeleb1 speaker verification evaluation benchmark, achieving Equal Error Rate 1.29%, 1.60%, and 2.80% for trial VoxCeleb1-{O,E,H} respectively. These results demonstrate relative improvements of 28.3%, 19.6%, and 22.6% over the current best self-supervised methods, thereby advancing the frontiers of SV technology.