mlsanityData to 7 Oct 2026

Imperceptible Rhythm Backdoor Attacks: Exploring Rhythm Transformation for Embedding Undetectable Vulnerabilities on Speech Recognition

arXiv 2406.10932 ↗First posted 16 Jun 2024cs.SD cs.AI eess.AScited by 5

On the map: LLM safety and security → Audio jailbreaks and speech backdoors

Abstract (arXiv)

Speech recognition is an essential start ring of human-computer interaction, and recently, deep learning models have achieved excellent success in this task. However, when the model training and private data provider are always separated, some security threats that make deep neural networks (DNNs) abnormal deserve to be researched. In recent years, the typical backdoor attacks have been researched in speech recognition systems. The existing backdoor methods are based on data poisoning. The attacker adds some incorporated changes to benign speech spectrograms or changes the speech components, such as pitch and timbre. As a result, the poisoned data can be detected by human hearing or automatic deep algorithms. To improve the stealthiness of data poisoning, we propose a non-neural and fast algorithm called Random Spectrogram Rhythm Transformation (RSRT) in this paper. The algorithm combines four steps to generate stealthy poisoned utterances. From the perspective of rhythm component transformation, our proposed trigger stretches or squeezes the mel spectrograms and recovers them back to signals. The operation keeps timbre and content unchanged for good stealthiness. Our experiments are conducted on two kinds of speech recognition tasks, including testing the stealthiness of poisoned samples by speaker verification and automatic speech recognition. The results show that our method has excellent effectiveness and stealthiness. The rhythm trigger needs a low poisoning rate and gets a very high attack success rate.

Key facts. Cited by 5 papers in the map (data through 7 Oct 2026). Placed in Audio jailbreaks and speech backdoors, one of 12 directions in LLM safety and security. Builds on 48 of its 78 references that are in the map. First posted on arXiv 16 Jun 2024.

Builds on

48 of 78 references.

Closest ancestors · 4

Same direction, newest first

  1. 2023Towards Stealthy Backdoor Attacks against Speech Recognition via Elements of Sound17 Jul
  2. 2023Fake the Real: Backdoor Attack on Deep Speech Classification via Voice Conversion28 Jun
  3. 2022VSVC: Backdoor attack against Keyword Spotting based on Voiceprint Selection and Voice Conversion20 Dec
  4. 2022PBSM: Backdoor attack against Keyword spotting based on pitch boosting and sound masking16 Nov

Other references · 44

  1. 2015Deep Residual Learning for Image Recognitioncited by 51,809
  2. 2020HiFi-GAN: Generative Adversarial Networks for Efficient and High Fidelity Speech Synthesiscited by 1,139
  3. 2017Targeted Backdoor Attacks on Deep Learning Systems Using Data Poisoningcited by 1,012
  4. 2016CNN Architectures for Large-Scale Audio Classificationcited by 701
  5. 2018Spectral Signatures in Backdoor Attackscited by 457
  6. 2020Backdoor Learning: A Surveycited by 412
  7. 2019Hidden Trigger Backdoor Attackscited by 359
  8. 2020Reflection Backdoor: A Natural Backdoor Attack on Deep Neural Networkscited by 341
  9. 2020Input-Aware Dynamic Backdoor Attackcited by 304
  10. 2019Label-Consistent Backdoor Attackscited by 285
  11. 2017A Dual-Stage Attention-Based Recurrent Neural Network for Time Series Predictioncited by 245
  12. 2019A backdoor attack against LSTM-based text classification systemscited by 214
32 more
  1. 2020Blind Backdoors in Deep Learning Modelscited by 174
  2. 2020Clean-Label Backdoor Attacks on Video Recognition Modelscited by 162
  3. 2017Neural Trojanscited by 151
  4. 2018SentiNet: Detecting Localized Universal Attacks Against Deep Learning Systemscited by 148
  5. 2022Paraformer: Fast and Accurate Parallel Transformer for Non-autoregressive End-to-End Speech Recognitioncited by 139
  6. 2020Backdoor Attacks Against Deep Learning Systems in the Physical Worldcited by 129
  7. 2020Backdoor Attacks and Countermeasures on Deep Learning: A Comprehensive Reviewcited by 120
  8. 2021Emotional Voice Conversion: Theory, Databases and ESDcited by 113
  9. 2023CAM++: A Fast and Efficient Network for Speaker Verification Using Context-Aware Maskingcited by 95
  10. 2020Deep Feature Space Trojan Attack of Neural Networks by Controlled Detoxificationcited by 93
  11. 2020Unsupervised Speech Decomposition via Triple Information Bottleneckcited by 87
  12. 2021Backdoor Attack in the Physical Worldcited by 82
  13. 2020Mitigating backdoor attacks in LSTM-based Text Classification Systems by Backdoor Keyword Identificationcited by 80
  14. 2021Black-box Detection of Backdoor Attacks with Limited Information and Datacited by 71
  15. 2021VQMIVC: Vector Quantization and Mutual Information-Based Unsupervised Speech Representation Disentanglement for One-shot Voice Conversioncited by 63
  16. 2020Backdoor Attack against Speaker Verificationcited by 62
  17. 2021Keyword Transformer: A Self-Attention Model for Keyword Spottingcited by 59
  18. 2021Poison Ink: Robust and Invisible Backdoor Attackcited by 50
  19. 2020Rethinking CNN Models for Audio Classificationcited by 48
  20. 2021Can You Hear It? Backdoor Attacks via Ultrasonic Triggerscited by 41
  21. 2021An Overview of Backdoor Attacks Against Deep Neural Networks and Possible Defencescited by 41
  22. 2023An Enhanced Res2Net with Local and Global Feature Fusion for Speaker Verificationcited by 41
  23. 2020ConFoc: Content-Focus Protection Against Trojan Attacks on Neural Networkscited by 28
  24. 2022SpeechSplit 2.0: Unsupervised speech disentanglement for voice conversion Without tuning autoencoder Bottleneckscited by 27
  25. 2018Attention Based Fully Convolutional Network for Speech Emotion Recognitioncited by 23
  26. 2020FaceHack: Triggering backdoored facial recognition systems using facial characteristicscited by 19
  27. 2022Going In Style: Audio Backdoors Through Stylistic Transformationscited by 19
  28. 2023Versatile Backdoor Attack with Visible, Semantic, Sample-Specific, and Compatible Triggerscited by 8
  29. 2019Adversarial Audio: A New Information Hiding Method and Backdoor for DNN-based Speech Recognition Modelscited by 7
  30. 2024Backdoor Attacks and Defenses on Semantic-Symbol Reconstruction in Semantic Communicationscited by 5
  31. 2023Is It Possible to Backdoor Face Forgery Detection with Natural Triggers?cited by 4
  32. 2024An Invisible Backdoor Attack Based On Semantic Featurecited by 2

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2025: 2; 2026: 3.

5 papers.

  1. 2025Investigating Vulnerabilities and Defenses Against Audio-Visual Attacks: A Comprehensive Survey Emphasizing Multimodal Modelscited by 1
  2. 2025SPBA: Utilizing Speech Large Language Model for Backdoor Attacks on Speech Classification Modelscited by 0
  3. 2026STEP: Detecting Audio Backdoor Attacks via Stability-based Trigger Exposure Profilingcited by 0
  4. 2026Backdoor Attacks on Speech Emotion Recognition via TTS-Generated Poisoningcited by 0
  5. 2026Pmeta-TLA: Backdoor Attacks for Speech Classification Models via Meta-Learning with Timbre Leakage Attackcited by 0