# Within-sample variability-invariant loss for robust speaker recognition under noisy environments

arXiv: https://arxiv.org/abs/2002.00924 · HTML: https://mlsanity.com/paper/2002.00924/

## Key facts

- Cited by 5 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 3 Feb 2020.

## Built on by (most cited first, 5 of 5)

- [Dual-Branch Knowledge Distillation for Noise-Robust Synthetic Speech Detection](https://mlsanity.com/paper/2310.08869/) (2023)
- [Learning Repeatable Speech Embeddings Using An Intra-class Correlation Regularizer](https://mlsanity.com/paper/2310.17049/) (2023)
- [Diff-SV: A Unified Hierarchical Framework for Noise-Robust Speaker Verification Using Score-Based Diffusion Probabilistic Models](https://mlsanity.com/paper/2309.08320/) (2023)
- [Self-supervised Reflective Learning through Self-distillation and Online Clustering for Speaker Representation Learning](https://mlsanity.com/paper/2401.01473/) (2024)
- [Noise-Conditioned Mixture-of-Experts Framework for Robust Speaker Verification](https://mlsanity.com/paper/2510.18533/) (2025)

## Abstract (arXiv)

Despite the significant improvements in speaker recognition enabled by deep neural networks, unsatisfactory performance persists under noisy environments. In this paper, we train the speaker embedding network to learn the "clean" embedding of the noisy utterance. Specifically, the network is trained with the original speaker identification loss with an auxiliary within-sample variability-invariant loss. This auxiliary variability-invariant loss is used to learn the same embedding among the clean utterance and its noisy copies and prevents the network from encoding the undesired noises or variabilities into the speaker representation. Furthermore, we investigate the data preparation strategy for generating clean and noisy utterance pairs on-the-fly. The strategy generates different noisy copies for the same clean utterance at each training step, helping the speaker embedding network generalize better under noisy environments. Experiments on VoxCeleb1 indicate that the proposed training framework improves the performance of the speaker verification system in both clean and noisy conditions.