# Self-supervised Speaker Recognition with Loss-gated Learning

arXiv: https://arxiv.org/abs/2110.03869 · HTML: https://mlsanity.com/paper/2110.03869/

## Key facts

- Cited by 14 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 8 Oct 2021.

## Built on by (most cited first, 14 of 14)

- [Pushing the limits of raw waveform speaker recognition](https://mlsanity.com/paper/2203.08488/) (2022)
- [Disentangling Voice and Content with Self-Supervision for Speaker Recognition](https://mlsanity.com/paper/2310.01128/) (2023)
- [Learning Lip-Based Audio-Visual Speaker Embeddings with AV-HuBERT](https://mlsanity.com/paper/2205.07180/) (2022)
- [Revealing Emotional Clusters in Speaker Embeddings: A Contrastive Learning Strategy for Speech Emotion Recognition](https://mlsanity.com/paper/2401.11017/) (2024)
- [Invisible Backdoor Attacks Using Data Poisoning in the Frequency Domain](https://mlsanity.com/paper/2207.04209/) (2022)
- [Self-Distillation Prototypes Network: Learning Robust Speaker Representations without Supervision](https://mlsanity.com/paper/2406.11169/) (2024)
- [Emphasized Non-Target Speaker Knowledge in Knowledge Distillation for Automatic Speaker Verification](https://mlsanity.com/paper/2309.14838/) (2023)
- [Towards Supervised Performance on Speaker Verification with Self-Supervised Learning by Leveraging Large-Scale ASR Models](https://mlsanity.com/paper/2406.02285/) (2024)
- [Self-Supervised Frameworks for Speaker Verification via Bootstrapped Positive Sampling](https://mlsanity.com/paper/2501.17772/) (2025)
- [Self-supervised Reflective Learning through Self-distillation and Online Clustering for Speaker Representation Learning](https://mlsanity.com/paper/2401.01473/) (2024)
- [Interpolating Speaker Identities in Embedding Space for Data Expansion](https://mlsanity.com/paper/2508.19210/) (2025)
- [Cluster-Guided Unsupervised Domain Adaptation for Deep Speaker Embedding](https://mlsanity.com/paper/2303.15944/) (2023)
- [Self-Distillation Prototypes Network: Learning Robust Speaker Representations without Supervision](https://mlsanity.com/paper/2308.02774/) (2023)
- [Self-Supervised Learning for Speaker Recognition: A study and review](https://mlsanity.com/paper/2602.10829/) (2026)

## Abstract (arXiv)

In self-supervised learning for speaker recognition, pseudo labels are useful as the supervision signals. It is a known fact that a speaker recognition model doesn't always benefit from pseudo labels due to their unreliability. In this work, we observe that a speaker recognition network tends to model the data with reliable labels faster than those with unreliable labels. This motivates us to study a loss-gated learning (LGL) strategy, which extracts the reliable labels through the fitting ability of the neural network during training. With the proposed LGL, our speaker recognition model obtains a 46.3% performance gain over the system without it. Further, the proposed self-supervised speaker recognition with LGL trained on the VoxCeleb2 dataset without any labels achieves an equal error rate of 1.66% on the VoxCeleb1 original test set. Code has been made available at: https://github.com/TaoRuijie/Loss-Gated-Learning.