# Exploring Targeted Universal Adversarial Perturbations to End-to-end ASR Models

arXiv: https://arxiv.org/abs/2104.02757 · HTML: https://mlsanity.com/paper/2104.02757/

## Key facts

- Cited by 10 papers in the map (data through 14 Oct 2026).
- Builds on 21 of its 27 references that are in the map.
- First posted on arXiv 6 Apr 2021.

## Builds on (most cited first, 21 of 21)

- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [Explaining and Harnessing Adversarial Examples](https://mlsanity.com/paper/1412.6572/) (2014)
- [Intriguing properties of neural networks](https://mlsanity.com/paper/1312.6199/) (2013)
- [Towards Evaluating the Robustness of Neural Networks](https://mlsanity.com/paper/1608.04644/) (2016)
- [Conformer: Convolution-augmented Transformer for Speech Recognition](https://mlsanity.com/paper/2005.08100/) (2020)
- [SpecAugment: A Simple Data Augmentation Method for Automatic Speech Recognition](https://mlsanity.com/paper/1904.08779/) (2019)
- [Universal adversarial perturbations](https://mlsanity.com/paper/1610.08401/) (2016)
- [Adversarial Examples for Evaluating Reading Comprehension Systems](https://mlsanity.com/paper/1707.07328/) (2017)
- [Adversarial Patch](https://mlsanity.com/paper/1712.09665/) (2017)
- [Generating Natural Language Adversarial Examples](https://mlsanity.com/paper/1804.07998/) (2018)
- [Audio Adversarial Examples: Targeted Attacks on Speech-to-Text](https://mlsanity.com/paper/1801.01944/) (2018)
- [Imperceptible, Robust, and Targeted Adversarial Examples for Automatic Speech Recognition](https://mlsanity.com/paper/1903.10346/) (2019)
- [Lingvo: a Modular and Scalable Framework for Sequence-to-Sequence Modeling](https://mlsanity.com/paper/1902.08295/) (2019)
- [Houdini: Fooling Deep Structured Prediction Models](https://mlsanity.com/paper/1707.05373/) (2017)
- [Did you hear that? Adversarial Examples Against Automatic Speech Recognition](https://mlsanity.com/paper/1801.00554/) (2018)
- [SoK: The Faults in our ASRs: An Overview of Attacks against Automatic Speech Recognition and Speaker Identification Systems](https://mlsanity.com/paper/2007.06622/) (2020)
- [Universal Adversarial Perturbations for Speech Recognition Systems](https://mlsanity.com/paper/1905.03828/) (2019)
- [Practical Hidden Voice Attacks against Speech and Speaker Recognition Systems](https://mlsanity.com/paper/1904.05734/) (2019)
- [Real-time, Universal, and Robust Adversarial Attacks Against Speaker Recognition Systems](https://mlsanity.com/paper/2003.02301/) (2020)
- [Universal Adversarial Audio Perturbations](https://mlsanity.com/paper/1908.03173/) (2019)
- [Universal adversarial examples in speech command classification](https://mlsanity.com/paper/1911.10182/) (2019)

## Built on by (most cited first, 10 of 10)

- [Universal and Transferable Adversarial Attacks on Aligned Language Models](https://mlsanity.com/paper/2307.15043/) (2023)
- [Neural Polarizer: A Lightweight and Effective Backdoor Defense via Purifying Poisoned Features](https://mlsanity.com/paper/2306.16697/) (2023)
- [Recent improvements of ASR models in the face of adversarial attacks](https://mlsanity.com/paper/2203.16536/) (2022)
- [Muting Whisper: A Universal Acoustic Adversarial Attack on Speech Foundation Models](https://mlsanity.com/paper/2405.06134/) (2024)
- [There is more than one kind of robustness: Fooling Whisper with adversarial examples](https://mlsanity.com/paper/2210.17316/) (2022)
- [Real-Time Neural Voice Camouflage](https://mlsanity.com/paper/2112.07076/) (2021)
- [Universal Acoustic Adversarial Attacks for Flexible Control of Speech-LLMs](https://mlsanity.com/paper/2505.14286/) (2025)
- [Watch What You Pretrain For: Targeted, Transferable Adversarial Examples on Self-Supervised Speech Recognition models](https://mlsanity.com/paper/2209.13523/) (2022)
- [Controlling Whisper: Universal Acoustic Adversarial Attacks to Control Speech Foundation Models](https://mlsanity.com/paper/2407.04482/) (2024)
- [Adversarial Attacks on ASR Systems: An Overview](https://mlsanity.com/paper/2208.02250/) (2022)

## Abstract (arXiv)

Although end-to-end automatic speech recognition (e2e ASR) models are widely deployed in many applications, there have been very few studies to understand models' robustness against adversarial perturbations. In this paper, we explore whether a targeted universal perturbation vector exists for e2e ASR models. Our goal is to find perturbations that can mislead the models to predict the given targeted transcript such as "thank you" or empty string on any input utterance. We study two different attacks, namely additive and prepending perturbations, and their performances on the state-of-the-art LAS, CTC and RNN-T models. We find that LAS is the most vulnerable to perturbations among the three models. RNN-T is more robust against additive perturbations, especially on long utterances. And CTC is robust against both additive and prepending perturbations. To attack RNN-T, we find prepending perturbation is more effective than the additive perturbation, and can mislead the models to predict the same short target on utterances of arbitrary length.