# EmoAttack: Utilizing Emotional Voice Conversion for Speech Backdoor Attacks on Deep Speech Classification Models

arXiv: https://arxiv.org/abs/2408.15508 · HTML: https://mlsanity.com/paper/2408.15508/

## Key facts

- Cited by 2 papers in the map (data through 14 Oct 2026).
- Placed in Audio jailbreaks and speech backdoors, one of 12 directions in LLM safety and security.
- Builds on 24 of its 33 references that are in the map.
- First posted on arXiv 28 Aug 2024.

## Placed in

- LLM safety and security / [Audio jailbreaks and speech backdoors](https://mlsanity.com/safety/audio-jailbreaks-and-speech-backdoors/)

## Builds on (most cited first, 24 of 24)

- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [Very Deep Convolutional Networks for Large-Scale Image Recognition](https://mlsanity.com/paper/1409.1556/) (2014)
- [BadNets: Identifying Vulnerabilities in the Machine Learning Model Supply Chain](https://mlsanity.com/paper/1708.06733/) (2017)
- [Speech Commands: A Dataset for Limited-Vocabulary Speech Recognition](https://mlsanity.com/paper/1804.03209/) (2018)
- [VoxCeleb: a large-scale speaker identification dataset](https://mlsanity.com/paper/1706.08612/) (2017)
- [Backdoor Learning: A Survey](https://mlsanity.com/paper/2007.08745/) (2020)
- [Generalized End-to-End Loss for Speaker Verification](https://mlsanity.com/paper/1710.10467/) (2017)
- [A Dual-Stage Attention-Based Recurrent Neural Network for Time Series Prediction](https://mlsanity.com/paper/1704.02971/) (2017)
- [A backdoor attack against LSTM-based text classification systems](https://mlsanity.com/paper/1905.12457/) (2019)
- [Blind Backdoors in Deep Learning Models](https://mlsanity.com/paper/2005.03823/) (2020)
- [Speaker Recognition from Raw Waveform with SincNet](https://mlsanity.com/paper/1808.00158/) (2018)
- [Emotional Voice Conversion: Theory, Databases and ESD](https://mlsanity.com/paper/2105.14762/) (2021)
- [Mitigating backdoor attacks in LSTM-based Text Classification Systems by Backdoor Keyword Identification](https://mlsanity.com/paper/2007.12070/) (2020)
- [Backdoor Attack against Speaker Verification](https://mlsanity.com/paper/2010.11607/) (2020)
- [Keyword Transformer: A Self-Attention Model for Keyword Spotting](https://mlsanity.com/paper/2104.00769/) (2021)
- [Can You Hear It? Backdoor Attacks via Ultrasonic Triggers](https://mlsanity.com/paper/2107.14569/) (2021)
- [Towards Stealthy Backdoor Attacks against Speech Recognition via Elements of Sound](https://mlsanity.com/paper/2307.08208/) (2023)
- [Natural Backdoor Attack on Text Data](https://mlsanity.com/paper/2006.16176/) (2020)
- [Going In Style: Audio Backdoors Through Stylistic Transformations](https://mlsanity.com/paper/2211.03117/) (2022)
- [Fake the Real: Backdoor Attack on Deep Speech Classification via Voice Conversion](https://mlsanity.com/paper/2306.15875/) (2023)
- [An Improved StarGAN for Emotional Voice Conversion: Enhancing Voice Quality and Data Augmentation](https://mlsanity.com/paper/2107.08361/) (2021)
- [VSVC: Backdoor attack against Keyword Spotting based on Voiceprint Selection and Voice Conversion](https://mlsanity.com/paper/2212.10103/) (2022)
- [PBSM: Backdoor attack against Keyword spotting based on pitch boosting and sound masking](https://mlsanity.com/paper/2211.08697/) (2022)
- [SER Evals: In-domain and Out-of-domain Benchmarking for Speech Emotion Recognition](https://mlsanity.com/paper/2408.07851/) (2024)

## Built on by (most cited first, 2 of 2)

- [Investigating Vulnerabilities and Defenses Against Audio-Visual Attacks: A Comprehensive Survey Emphasizing Multimodal Models](https://mlsanity.com/paper/2506.11521/) (2025)
- [STEP: Detecting Audio Backdoor Attacks via Stability-based Trigger Exposure Profiling](https://mlsanity.com/paper/2603.18103/) (2026)

## Abstract (arXiv)

Deep speech classification tasks, mainly including keyword spotting and speaker verification, play a crucial role in speech-based human-computer interaction. Recently, the security of these technologies has been demonstrated to be vulnerable to backdoor attacks. Specifically speaking, speech samples are attacked by noisy disruption and component modification in present triggers. We suggest that speech backdoor attacks can strategically focus on emotion, a higher-level subjective perceptual attribute inherent in speech. Furthermore, we proposed that emotional voice conversion technology can serve as the speech backdoor attack trigger, and the method is called EmoAttack. Based on this, we conducted attack experiments on two speech classification tasks, showcasing that EmoAttack method owns impactful trigger effectiveness and its remarkable attack success rate and accuracy variance. Additionally, the ablation experiments found that speech with intensive emotion is more suitable to be targeted for attacks.