Clean Label Attacks against SLU Systems
On the map: LLM safety and security → Audio jailbreaks and speech backdoors
Abstract (arXiv)
Poisoning backdoor attacks involve an adversary manipulating the training data to induce certain behaviors in the victim model by inserting a trigger in the signal at inference time. We adapted clean label backdoor (CLBD)-data poisoning attacks, which do not modify the training labels, on state-of-the-art speech recognition models that support/perform a Spoken Language Understanding task, achieving 99.8% attack success rate by poisoning 10% of the training data. We analyzed how varying the signal-strength of the poison, percent of samples poisoned, and choice of trigger impact the attack. We also found that CLBD attacks are most successful when applied to training samples that are inherently hard for a proxy model. Using this strategy, we achieved an attack success rate of 99.3% by poisoning a meager 1.5% of the training data. Finally, we applied two previously developed defenses against gradient-based attacks, and found that they attain mixed success against poisoning.
Key facts. Cited by 2 papers in the map (data through 7 Oct 2026). Placed in Audio jailbreaks and speech backdoors, one of 12 directions in LLM safety and security. Builds on 20 of its 41 references that are in the map. First posted on arXiv 13 Sep 2024.
Builds on
20 of 41 references.
References · 20
- 2020Language Models are Few-Shot Learnerscited by 32,199
- 2017Towards Deep Learning Models Resistant to Adversarial Attackscited by 6,326
- 2021Convex optimizationcited by 2,479
- 2020Conformer: Convolution-augmented Transformer for Speech Recognitioncited by 1,838
- 2017Targeted Backdoor Attacks on Deep Learning Systems Using Data Poisoningcited by 1,012
- 2018VoxCeleb2: Deep Speaker Recognitioncited by 862
- 2017Mitigating Adversarial Effects Through Randomizationcited by 382
- 2019Label-Consistent Backdoor Attackscited by 285
- 2023Poisoning Web-Scale Training Datasets is Practicalcited by 276
- 2019Speech Model Pre-training for End-to-End Spoken Language Understandingcited by 191
- 2022Pruned RNN-T for fast, memory-efficient ASR trainingcited by 46
- 2023Reproducing Whisper-Style Training Using an Open-Source Toolkit and Publicly Available Datacited by 42
8 more
- 2022Not All Poisons are Created Equal: Robust Training against Data Poisoningcited by 29
- 2021What Doesn't Kill You Makes You Robust(er): How to Adversarially Train against Data Poisoningcited by 22
- 2021Closer Look at the Transferability of Adversarial Examples: How They Fool Different Models Differentlycited by 22
- 2019Adversarial Music: Real World Audio Adversary Against Wake-word Detection Systemcited by 20
- 2021Putting words into the system's mouth: A targeted attack on neural machine translation using monolingual data poisoningcited by 19
- 2023Forcing Generative Models to Degenerate Ones: The Power of Data Poisoning Attackscited by 9
- 2022Defense against Adversarial Attacks on Hybrid Speech Recognition using Joint Adversarial Fine-tuning with Denoisercited by 3
- 2024Unraveling Adversarial Examples against Speaker Identification -- Techniques for Attack Detection and Victim Model Classificationcited by 1
Built on by
Citing papers per year: 2025: 1; 2026: 1.
2 papers.