# Self-Supervised Speaker Verification with Simple Siamese Network and Self-Supervised Regularization

arXiv: https://arxiv.org/abs/2112.04459 · HTML: https://mlsanity.com/paper/2112.04459/

## Key facts

- Cited by 17 papers in the map (data through 14 Oct 2026).
- Builds on 23 of its 27 references that are in the map.
- First posted on arXiv 8 Dec 2021.

## Builds on (most cited first, 23 of 23)

- [A Simple Framework for Contrastive Learning of Visual Representations](https://mlsanity.com/paper/2002.05709/) (2020)
- [Momentum Contrast for Unsupervised Visual Representation Learning](https://mlsanity.com/paper/1911.05722/) (2019)
- [Bootstrap your own latent: A new approach to self-supervised Learning](https://mlsanity.com/paper/2006.07733/) (2020)
- [Exploring Simple Siamese Representation Learning](https://mlsanity.com/paper/2011.10566/) (2020)
- [SpecAugment: A Simple Data Augmentation Method for Automatic Speech Recognition](https://mlsanity.com/paper/1904.08779/) (2019)
- [CosFace: Large Margin Cosine Loss for Deep Face Recognition](https://mlsanity.com/paper/1801.09414/) (2018)
- [VoxCeleb2: Deep Speaker Recognition](https://mlsanity.com/paper/1806.05622/) (2018)
- [VoxCeleb: a large-scale speaker identification dataset](https://mlsanity.com/paper/1706.08612/) (2017)
- [ECAPA-TDNN: Emphasized Channel Attention, Propagation and Aggregation in TDNN Based Speaker Verification](https://mlsanity.com/paper/2005.07143/) (2020)
- [MUSAN: A Music, Speech, and Noise Corpus](https://mlsanity.com/paper/1510.08484/) (2015)
- [Generalized End-to-End Loss for Speaker Verification](https://mlsanity.com/paper/1710.10467/) (2017)
- [In defence of metric learning for speaker recognition](https://mlsanity.com/paper/2003.11982/) (2020)
- [Attentive Statistics Pooling for Deep Speaker Embedding](https://mlsanity.com/paper/1803.10963/) (2018)
- [Exploring the Encoding Layer and Loss Function in End-to-End Speaker and Language Recognition System](https://mlsanity.com/paper/1804.05160/) (2018)
- [Self-supervised Text-independent Speaker Verification using Prototypical Momentum Contrastive Learning](https://mlsanity.com/paper/2012.07178/) (2020)
- [Disentangled Speech Embeddings using Cross-modal Self-supervision](https://mlsanity.com/paper/2002.08742/) (2020)
- [Augmentation adversarial training for self-supervised speaker recognition](https://mlsanity.com/paper/2007.12085/) (2020)
- [Seeing voices and hearing voices: learning discriminative embeddings using cross-modal self-supervision](https://mlsanity.com/paper/2004.14326/) (2020)
- [Semi-Supervised Contrastive Learning with Generalized Contrastive Loss and Its Application to Speaker Recognition](https://mlsanity.com/paper/2006.04326/) (2020)
- [DEAAN: Disentangled Embedding and Adversarial Adaptation Network for Robust Speaker Representation Learning](https://mlsanity.com/paper/2012.06896/) (2020)
- [Generative Adversarial Speaker Embedding Networks for Domain Robust End-to-End Speaker Verification](https://mlsanity.com/paper/1811.03063/) (2018)
- [Open-set Short Utterance Forensic Speaker Verification using Teacher-Student Network with Explicit Inductive Bias](https://mlsanity.com/paper/2009.09556/) (2020)
- [Learning Speaker Embedding with Momentum Contrast](https://mlsanity.com/paper/2001.01986/) (2020)

## Built on by (most cited first, 17 of 17)

- [Pushing the limits of raw waveform speaker recognition](https://mlsanity.com/paper/2203.08488/) (2022)
- [Singer Identity Representation Learning using Self-Supervised Techniques](https://mlsanity.com/paper/2401.05064/) (2024)
- [Multi-Frequency Information Enhanced Channel Attention Module for Speaker Representation Learning](https://mlsanity.com/paper/2207.04540/) (2022)
- [Self-Distillation Prototypes Network: Learning Robust Speaker Representations without Supervision](https://mlsanity.com/paper/2406.11169/) (2024)
- [Improving Transformer-based Networks With Locality For Automatic Speaker Verification](https://mlsanity.com/paper/2302.08639/) (2023)
- [Efficient Adapter Tuning of Pre-trained Speech Models for Automatic Speaker Verification](https://mlsanity.com/paper/2403.00293/) (2024)
- [Self-Supervised Frameworks for Speaker Verification via Bootstrapped Positive Sampling](https://mlsanity.com/paper/2501.17772/) (2025)
- [Weak Alignment Supervision from Hybrid Model Improves End-to-end ASR](https://mlsanity.com/paper/2311.14835/) (2023)
- [Toward Improving Synthetic Audio Spoofing Detection Robustness via Meta-Learning and Disentangled Training With Adversarial Examples](https://mlsanity.com/paper/2408.13341/) (2024)
- [JOOCI: a Framework for Learning Comprehensive Speech Representations](https://mlsanity.com/paper/2410.11086/) (2024)
- [UniPET-SPK: A Unified Framework for Parameter-Efficient Tuning of Pre-trained Speech Models for Robust Speaker Verification](https://mlsanity.com/paper/2501.16542/) (2025)
- [Pushing the Frontiers of Self-Distillation Prototypes Network with Dimension Regularization and Score Normalization](https://mlsanity.com/paper/2505.13826/) (2025)
- [SSPS: Self-Supervised Positive Sampling for Robust Self-Supervised Speaker Verification](https://mlsanity.com/paper/2505.14561/) (2025)
- [Dynamic Recognition of Speakers for Consent Management by Contrastive Embedding Replay](https://mlsanity.com/paper/2205.08459/) (2022)
- [Self-Distillation Prototypes Network: Learning Robust Speaker Representations without Supervision](https://mlsanity.com/paper/2308.02774/) (2023)
- [Speech Representation Learning Revisited: The Necessity of Separate Learnable Parameters and Robust Data Augmentation](https://mlsanity.com/paper/2408.10557/) (2024)
- [Self-Supervised Learning for Speaker Recognition: A study and review](https://mlsanity.com/paper/2602.10829/) (2026)

## Abstract (arXiv)

Training speaker-discriminative and robust speaker verification systems without speaker labels is still challenging and worthwhile to explore. In this study, we propose an effective self-supervised learning framework and a novel regularization strategy to facilitate self-supervised speaker representation learning. Different from contrastive learning-based self-supervised learning methods, the proposed self-supervised regularization (SSReg) focuses exclusively on the similarity between the latent representations of positive data pairs. We also explore the effectiveness of alternative online data augmentation strategies on both the time domain and frequency domain. With our strong online data augmentation strategy, the proposed SSReg shows the potential of self-supervised learning without using negative pairs and it can significantly improve the performance of self-supervised speaker representation learning with a simple Siamese network architecture. Comprehensive experiments on the VoxCeleb datasets demonstrate that our proposed self-supervised approach obtains a 23.4% relative improvement by adding the effective self-supervised regularization and outperforms other previous works.