# Clean Label Attacks against SLU Systems

arXiv: https://arxiv.org/abs/2409.08985 · HTML: https://mlsanity.com/paper/2409.08985/

## Key facts

- Cited by 2 papers in the map (data through 14 Oct 2026).
- Placed in Audio jailbreaks and speech backdoors, one of 12 directions in LLM safety and security.
- Builds on 20 of its 41 references that are in the map.
- First posted on arXiv 13 Sep 2024.

## Placed in

- LLM safety and security / [Audio jailbreaks and speech backdoors](https://mlsanity.com/safety/audio-jailbreaks-and-speech-backdoors/)

## Builds on (most cited first, 20 of 20)

- [Language Models are Few-Shot Learners](https://mlsanity.com/paper/2005.14165/) (2020)
- [Towards Deep Learning Models Resistant to Adversarial Attacks](https://mlsanity.com/paper/1706.06083/) (2017)
- [Convex optimization](https://mlsanity.com/paper/2106.01946/) (2021)
- [Conformer: Convolution-augmented Transformer for Speech Recognition](https://mlsanity.com/paper/2005.08100/) (2020)
- [Targeted Backdoor Attacks on Deep Learning Systems Using Data Poisoning](https://mlsanity.com/paper/1712.05526/) (2017)
- [VoxCeleb2: Deep Speaker Recognition](https://mlsanity.com/paper/1806.05622/) (2018)
- [Mitigating Adversarial Effects Through Randomization](https://mlsanity.com/paper/1711.01991/) (2017)
- [Label-Consistent Backdoor Attacks](https://mlsanity.com/paper/1912.02771/) (2019)
- [Poisoning Web-Scale Training Datasets is Practical](https://mlsanity.com/paper/2302.10149/) (2023)
- [Speech Model Pre-training for End-to-End Spoken Language Understanding](https://mlsanity.com/paper/1904.03670/) (2019)
- [Pruned RNN-T for fast, memory-efficient ASR training](https://mlsanity.com/paper/2206.13236/) (2022)
- [Reproducing Whisper-Style Training Using an Open-Source Toolkit and Publicly Available Data](https://mlsanity.com/paper/2309.13876/) (2023)
- [Not All Poisons are Created Equal: Robust Training against Data Poisoning](https://mlsanity.com/paper/2210.09671/) (2022)
- [What Doesn't Kill You Makes You Robust(er): How to Adversarially Train against Data Poisoning](https://mlsanity.com/paper/2102.13624/) (2021)
- [Closer Look at the Transferability of Adversarial Examples: How They Fool Different Models Differently](https://mlsanity.com/paper/2112.14337/) (2021)
- [Putting words into the system's mouth: A targeted attack on neural machine translation using monolingual data poisoning](https://mlsanity.com/paper/2107.05243/) (2021)
- [Adversarial Music: Real World Audio Adversary Against Wake-word Detection System](https://mlsanity.com/paper/1911.00126/) (2019)
- [Forcing Generative Models to Degenerate Ones: The Power of Data Poisoning Attacks](https://mlsanity.com/paper/2312.04748/) (2023)
- [Defense against Adversarial Attacks on Hybrid Speech Recognition using Joint Adversarial Fine-tuning with Denoiser](https://mlsanity.com/paper/2204.03851/) (2022)
- [Unraveling Adversarial Examples against Speaker Identification -- Techniques for Attack Detection and Victim Model Classification](https://mlsanity.com/paper/2402.19355/) (2024)

## Built on by (most cited first, 2 of 2)

- [Where Do Backdoors Live? A Component-Level Analysis of Backdoor Propagation in Speech Language Models](https://mlsanity.com/paper/2510.01157/) (2025)
- [DRL-CLBA: A Clean Label Backdoor Attack for Speech Classification via DDPG Reinforcement Learning](https://mlsanity.com/paper/2607.01729/) (2026)

## Abstract (arXiv)

Poisoning backdoor attacks involve an adversary manipulating the training data to induce certain behaviors in the victim model by inserting a trigger in the signal at inference time. We adapted clean label backdoor (CLBD)-data poisoning attacks, which do not modify the training labels, on state-of-the-art speech recognition models that support/perform a Spoken Language Understanding task, achieving 99.8% attack success rate by poisoning 10% of the training data. We analyzed how varying the signal-strength of the poison, percent of samples poisoned, and choice of trigger impact the attack. We also found that CLBD attacks are most successful when applied to training samples that are inherently hard for a proxy model. Using this strategy, we achieved an attack success rate of 99.3% by poisoning a meager 1.5% of the training data. Finally, we applied two previously developed defenses against gradient-based attacks, and found that they attain mixed success against poisoning.