mlsanityData to 7 Oct 2026

Where Do Backdoors Live? A Component-Level Analysis of Backdoor Propagation in Speech Language Models

arXiv 2510.01157 ↗First posted 1 Oct 2025cs.CL cs.CR cs.SDcited by 0

On the map: LLM safety and security → Backdoor and data poisoning attacks

Abstract (arXiv)

Speech language models (SLMs) are systems of systems: independent components that unite to achieve a common goal. Despite their heterogeneous nature, SLMs are often studied end-to-end; how information flows through the pipeline remains obscure. We investigate this question through the lens of backdoor attacks. We first establish that backdoors can propagate through the SLM, leaving all tasks highly vulnerable. From this, we design a component analysis to discover the role each component takes in backdoor learning. We find that backdoor persistence or erasure is highly dependent on the targeted component. Beyond propagation, we examine how backdoors are encoded in shared multitask embeddings, showing that poisoned samples are not directly separable from benign ones, challenging a common separability assumption used in filtering defenses. Our findings emphasize the need to treat multimodal pipelines as intricate systems with unique vulnerabilities, not solely extensions of unimodal ones.

Key facts. No citing papers in the map yet (data through 7 Oct 2026). Placed in Backdoor and data poisoning attacks, one of 12 directions in LLM safety and security. Builds on 37 of its 45 references that are in the map. First posted on arXiv 1 Oct 2025.

Builds on

37 of 45 references.

Closest ancestors · 2

Same direction, newest first

  1. 2025BadToken: Token-level Backdoor Attacks to Multi-modal Large Language Models20 Mar
  2. 2024Revisiting Backdoor Attacks against Large Vision-Language Models from Domain Shift27 Jun

Other references · 35

  1. 2020Language Models are Few-Shot Learnerscited by 32,199
  2. 2021LoRA: Low-Rank Adaptation of Large Language Modelscited by 14,084
  3. 2023LLaMA: Open and Efficient Foundation Language Modelscited by 13,958
  4. 2023Visual Instruction Tuningcited by 7,792
  5. 2023BLIP-2: Bootstrapping Language-Image Pre-training with Frozen Image Encoders and Large Language Modelscited by 6,069
  6. 2022Flamingo: a Visual Language Model for Few-Shot Learningcited by 4,337
  7. 2022Robust Speech Recognition via Large-Scale Weak Supervisioncited by 3,972
  8. 2020wav2vec 2.0: A Framework for Self-Supervised Learning of Speech Representationscited by 3,853
  9. 2023Qwen Technical Reportcited by 3,273
  10. 2021HuBERT: Self-Supervised Speech Representation Learning by Masked Prediction of Hidden Unitscited by 2,301
  11. 2021WavLM: Large-Scale Self-Supervised Pre-Training for Full Stack Speech Processingcited by 1,609
  12. 2017BadNets: Identifying Vulnerabilities in the Machine Learning Model Supply Chaincited by 1,052
23 more
  1. 2017Targeted Backdoor Attacks on Deep Learning Systems Using Data Poisoningcited by 1,012
  2. 2018VoxCeleb2: Deep Speaker Recognitioncited by 862
  3. 2024TinyLlama: An Open-Source Small Language Modelcited by 604
  4. 2023Qwen-Audio: Advancing Universal Audio Understanding via Unified Large-Scale Audio-Language Modelscited by 591
  5. 2023SpeechGPT: Empowering Large Language Models with Intrinsic Cross-Modal Conversational Abilitiescited by 518
  6. 2023SALMONN: Towards Generic Hearing Abilities for Large Language Modelscited by 492
  7. 2018Spectral Signatures in Backdoor Attackscited by 457
  8. 2018Detecting Backdoor Attacks on Deep Neural Networks by Activation Clusteringcited by 429
  9. 2019Hidden Trigger Backdoor Attackscited by 359
  10. 2023AudioPaLM: A Large Language Model That Can Speak and Listencited by 333
  11. 2021Adversarial Neuron Pruning Purifies Backdoored Deep Modelscited by 230
  12. 2022Backdoor Defense via Decoupling the Training Processcited by 149
  13. 2021SPECTRE: Defending Against Backdoor Attacks Using Robust Statisticscited by 123
  14. 2021Backdoor Attacks on Self-Supervised Learningcited by 89
  15. 2020Backdoor Attack against Speaker Verificationcited by 62
  16. 2023Enhancing Fine-Tuning Based Backdoor Defense with Sharpness-Aware Minimizationcited by 60
  17. 2023Reconstructive Neuron Pruning for Backdoor Defensecited by 56
  18. 2021Can You Hear It? Backdoor Attacks via Ultrasonic Triggerscited by 41
  19. 2022Data Poisoning Attacks Against Multimodal Encoderscited by 41
  20. 2023Towards Stealthy Backdoor Attacks against Speech Recognition via Elements of Soundcited by 28
  21. 2021VoxCeleb Enrichment for Age and Gender Recognitioncited by 20
  22. 2024Clean Label Attacks against SLU Systemscited by 2
  23. 2025Multi-Target Backdoor Attacks Against Speaker Recognitioncited by 1

Built on by

Not cited yet.