# Improving the Adversarial Robustness for Speaker Verification by Self-Supervised Learning

arXiv: https://arxiv.org/abs/2106.00273 · HTML: https://mlsanity.com/paper/2106.00273/

## Key facts

- Cited by 16 papers in the map (data through 14 Oct 2026).
- Builds on 60 of its 108 references that are in the map.
- First posted on arXiv 1 Jun 2021.

## Builds on (most cited first, 30 of 60)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [RoBERTa: A Robustly Optimized BERT Pretraining Approach](https://mlsanity.com/paper/1907.11692/) (2019)
- [Representation Learning with Contrastive Predictive Coding](https://mlsanity.com/paper/1807.03748/) (2018)
- [Explaining and Harnessing Adversarial Examples](https://mlsanity.com/paper/1412.6572/) (2014)
- [Intriguing properties of neural networks](https://mlsanity.com/paper/1312.6199/) (2013)
- [Squeeze-and-Excitation Networks](https://mlsanity.com/paper/1709.01507/) (2017)
- [ALBERT: A Lite BERT for Self-supervised Learning of Language Representations](https://mlsanity.com/paper/1909.11942/) (2019)
- [Adversarial examples in the physical world](https://mlsanity.com/paper/1607.02533/) (2016)
- [SpecAugment: A Simple Data Augmentation Method for Automatic Speech Recognition](https://mlsanity.com/paper/1904.08779/) (2019)
- [Adversarial Machine Learning at Scale](https://mlsanity.com/paper/1611.01236/) (2016)
- [VoxCeleb2: Deep Speaker Recognition](https://mlsanity.com/paper/1806.05622/) (2018)
- [The Limitations of Deep Learning in Adversarial Settings](https://mlsanity.com/paper/1511.07528/) (2015)
- [VoxCeleb: a large-scale speaker identification dataset](https://mlsanity.com/paper/1706.08612/) (2017)
- [ECAPA-TDNN: Emphasized Channel Attention, Propagation and Aggregation in TDNN Based Speaker Verification](https://mlsanity.com/paper/2005.07143/) (2020)
- [AISHELL-1: An Open-Source Mandarin Speech Corpus and A Speech Recognition Baseline](https://mlsanity.com/paper/1709.05522/) (2017)
- [Deep Speech: Scaling up end-to-end speech recognition](https://mlsanity.com/paper/1412.5567/) (2014)
- [vq-wav2vec: Self-Supervised Learning of Discrete Speech Representations](https://mlsanity.com/paper/1910.05453/) (2019)
- [Audio Adversarial Examples: Targeted Attacks on Speech-to-Text](https://mlsanity.com/paper/1801.01944/) (2018)
- [Mockingjay: Unsupervised Speech Representation Learning with Deep Bidirectional Transformer Encoders](https://mlsanity.com/paper/1910.12638/) (2019)
- [TERA: Self-Supervised Learning of Transformer Encoder Representation for Speech](https://mlsanity.com/paper/2007.06028/) (2020)
- [An Unsupervised Autoregressive Model for Speech Representation Learning](https://mlsanity.com/paper/1904.03240/) (2019)
- [A Self-supervised Approach for Adversarial Robustness](https://mlsanity.com/paper/2006.04924/) (2020)
- [Adversarial Robustness: From Self-Supervised Pre-Training to Fine-Tuning](https://mlsanity.com/paper/2003.12862/) (2020)
- [Imperceptible, Robust, and Targeted Adversarial Examples for Automatic Speech Recognition](https://mlsanity.com/paper/1903.10346/) (2019)
- [Unsupervised pretraining transfers well across languages](https://mlsanity.com/paper/2002.02848/) (2020)
- [Adversarial Attacks Against Automatic Speech Recognition Systems via Psychoacoustic Hiding](https://mlsanity.com/paper/1808.05665/) (2018)
- [Generative Pre-Training for Speech with Autoregressive Predictive Coding](https://mlsanity.com/paper/1910.12607/) (2019)
- [Effectiveness of self-supervised pre-training for speech recognition](https://mlsanity.com/paper/1911.03912/) (2019)

## Built on by (most cited first, 16 of 16)

- [Channel-wise Gated Res2Net: Towards Robust Detection of Synthetic Speech Attacks](https://mlsanity.com/paper/2107.08803/) (2021)
- [Partially Fake Audio Detection by Self-attention-based Fake Span Discovery](https://mlsanity.com/paper/2202.06684/) (2022)
- [Voting for the right answer: Adversarial defense for speaker verification](https://mlsanity.com/paper/2106.07868/) (2021)
- [The defender's perspective on automatic speaker verification: An overview](https://mlsanity.com/paper/2305.12804/) (2023)
- [Adversarial Sample Detection for Speaker Verification by Neural Vocoders](https://mlsanity.com/paper/2107.00309/) (2021)
- [Characterizing the adversarial vulnerability of speech self-supervised learning](https://mlsanity.com/paper/2111.04330/) (2021)
- [Certification of Speaker Recognition Models to Additive Perturbations](https://mlsanity.com/paper/2404.18791/) (2024)
- [Push-Pull: Characterizing the Adversarial Robustness for Audio-Visual Active Speaker Detection](https://mlsanity.com/paper/2210.00753/) (2022)
- [Mitigating Closed-model Adversarial Examples with Bayesian Neural Modeling for Enhanced End-to-End Speech Recognition](https://mlsanity.com/paper/2202.08532/) (2022)
- [Spoofing-Aware Speaker Verification by Multi-Level Fusion](https://mlsanity.com/paper/2203.15377/) (2022)
- [Adversarial Speaker Distillation for Countermeasure Model on Automatic Speaker Verification](https://mlsanity.com/paper/2203.17031/) (2022)
- [LMD: A Learnable Mask Network to Detect Adversarial Examples for Speaker Verification](https://mlsanity.com/paper/2211.00825/) (2022)
- [Tackling Spoofing-Aware Speaker Verification with Multi-Model Fusion](https://mlsanity.com/paper/2206.09131/) (2022)
- [An Experimental Study on Private Aggregation of Teacher Ensemble Learning for End-to-End Speech Recognition](https://mlsanity.com/paper/2210.05614/) (2022)
- [Impact of Phonetics on Speaker Identity in Adversarial Voice Attack](https://mlsanity.com/paper/2509.15437/) (2025)
- [Scalable Ensemble-based Detection Method against Adversarial Attacks for speaker verification](https://mlsanity.com/paper/2312.08622/) (2023)

## Abstract (arXiv)

Previous works have shown that automatic speaker verification (ASV) is seriously vulnerable to malicious spoofing attacks, such as replay, synthetic speech, and recently emerged adversarial attacks. Great efforts have been dedicated to defending ASV against replay and synthetic speech; however, only a few approaches have been explored to deal with adversarial attacks. All the existing approaches to tackle adversarial attacks for ASV require the knowledge for adversarial samples generation, but it is impractical for defenders to know the exact attack algorithms that are applied by the in-the-wild attackers. This work is among the first to perform adversarial defense for ASV without knowing the specific attack algorithms. Inspired by self-supervised learning models (SSLMs) that possess the merits of alleviating the superficial noise in the inputs and reconstructing clean samples from the interrupted ones, this work regards adversarial perturbations as one kind of noise and conducts adversarial defense for ASV by SSLMs. Specifically, we propose to perform adversarial defense from two perspectives: 1) adversarial perturbation purification and 2) adversarial perturbation detection. Experimental results show that our detection module effectively shields the ASV by detecting adversarial samples with an accuracy of around 80%. Moreover, since there is no common metric for evaluating the adversarial defense performance for ASV, this work also formalizes evaluation metrics for adversarial defense considering both purification and detection based approaches into account. We sincerely encourage future works to benchmark their approaches based on the proposed evaluation framework.