# Augmentation adversarial training for self-supervised speaker recognition

arXiv: https://arxiv.org/abs/2007.12085 · HTML: https://mlsanity.com/paper/2007.12085/

## Key facts

- Cited by 24 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 23 Jul 2020.

## Built on by (most cited first, 24 of 24)

- [SLUE: New Benchmark Tasks for Spoken Language Understanding Evaluation on Natural Speech](https://mlsanity.com/paper/2111.10367/) (2021)
- [Pushing the limits of raw waveform speaker recognition](https://mlsanity.com/paper/2203.08488/) (2022)
- [Synthetic Data in Human Analysis: A Survey](https://mlsanity.com/paper/2208.09191/) (2022)
- [Self-Supervised Speaker Verification with Simple Siamese Network and Self-Supervised Regularization](https://mlsanity.com/paper/2112.04459/) (2021)
- [Improving Distortion Robustness of Self-supervised Speech Processing Tasks with Domain Adaptation](https://mlsanity.com/paper/2203.16104/) (2022)
- [Non-Contrastive Self-supervised Learning for Utterance-Level Information Extraction from Speech](https://mlsanity.com/paper/2208.05445/) (2022)
- [Contrastive Neural Processes for Self-Supervised Learning](https://mlsanity.com/paper/2110.13623/) (2021)
- [VoxSRC 2022: The Fourth VoxCeleb Speaker Recognition Challenge](https://mlsanity.com/paper/2302.10248/) (2023)
- [The VoxCeleb Speaker Recognition Challenge: A Retrospective](https://mlsanity.com/paper/2408.14886/) (2024)
- [Improving generalizability of distilled self-supervised speech processing models under distorted settings](https://mlsanity.com/paper/2210.07978/) (2022)
- [Multi-Frequency Information Enhanced Channel Attention Module for Speaker Representation Learning](https://mlsanity.com/paper/2207.04540/) (2022)
- [Towards Supervised Performance on Speaker Verification with Self-Supervised Learning by Leveraging Large-Scale ASR Models](https://mlsanity.com/paper/2406.02285/) (2024)
- [Non-Contrastive Self-Supervised Learning of Utterance-Level Speech Representations](https://mlsanity.com/paper/2208.05413/) (2022)
- [Experimenting with Additive Margins for Contrastive Self-Supervised Speaker Verification](https://mlsanity.com/paper/2306.03664/) (2023)
- [Additive Margin in Contrastive Self-Supervised Frameworks to Learn Discriminative Speaker Representations](https://mlsanity.com/paper/2404.14913/) (2024)
- [ELP-Adapters: Parameter Efficient Adapter Tuning for Various Speech Processing Tasks](https://mlsanity.com/paper/2407.21066/) (2024)
- [Self-Supervised Frameworks for Speaker Verification via Bootstrapped Positive Sampling](https://mlsanity.com/paper/2501.17772/) (2025)
- [The JHU submission to VoxSRC-21: Track 3](https://mlsanity.com/paper/2109.13425/) (2021)
- [Bootstrap Equilibrium and Probabilistic Speaker Representation Learning for Self-supervised Speaker Verification](https://mlsanity.com/paper/2112.08929/) (2021)
- [Adversarial Data Augmentation for Robust Speaker Verification](https://mlsanity.com/paper/2402.02699/) (2024)
- [SSPS: Self-Supervised Positive Sampling for Robust Self-Supervised Speaker Verification](https://mlsanity.com/paper/2505.14561/) (2025)
- [Cluster-Guided Unsupervised Domain Adaptation for Deep Speaker Embedding](https://mlsanity.com/paper/2303.15944/) (2023)
- [Speech Representation Learning Revisited: The Necessity of Separate Learnable Parameters and Robust Data Augmentation](https://mlsanity.com/paper/2408.10557/) (2024)
- [Self-Supervised Learning for Speaker Recognition: A study and review](https://mlsanity.com/paper/2602.10829/) (2026)

## Abstract (arXiv)

The goal of this work is to train robust speaker recognition models without speaker labels. Recent works on unsupervised speaker representations are based on contrastive learning in which they encourage within-utterance embeddings to be similar and across-utterance embeddings to be dissimilar. However, since the within-utterance segments share the same acoustic characteristics, it is difficult to separate the speaker information from the channel information. To this end, we propose augmentation adversarial training strategy that trains the network to be discriminative for the speaker information, while invariant to the augmentation applied. Since the augmentation simulates the acoustic characteristics, training the network to be invariant to augmentation also encourages the network to be invariant to the channel information in general. Extensive experiments on the VoxCeleb and VOiCES datasets show significant improvements over previous works using self-supervision, and the performance of our self-supervised models far exceed that of humans.