mlsanityData to 14 Oct 2026

Breaking Audio Large Language Models by Attacking Only the Encoder: A Universal Targeted Latent-Space Audio Attack

arXiv 2512.23881 ↗First posted 29 Dec 2025cs.SD cs.AI cs.CRcited by 4

On the map: LLM reasoning → Audio-language model reasoning

Abstract (arXiv)

Audio-language models combine audio encoders with large language models to enable multimodal reasoning, but they also introduce new security vulnerabilities. We propose a universal targeted latent space attack, an encoder-level adversarial attack that manipulates audio latent representations to induce attacker-specified outputs in downstream language generation. Unlike prior waveform-level or input-specific attacks, our approach learns a universal perturbation that generalizes across inputs and speakers and does not require access to the language model. Experiments on Qwen2-Audio-7B-Instruct demonstrate consistently high attack success rates with minimal perceptual distortion, revealing a critical and previously underexplored attack surface at the encoder level of multimodal systems.

Key facts. Cited by 4 papers in the map (data through 14 Oct 2026). Placed in Audio-language model reasoning, one of 17 directions in LLM reasoning. Builds on 41 of its 46 references that are in the map. First posted on arXiv 29 Dec 2025.

Builds on

41 of 46 references.

References · 41

  1. 2014Adam: A Method for Stochastic Optimizationcited by 37,240
  2. 2014Explaining and Harnessing Adversarial Examplescited by 7,364
  3. 2017Towards Deep Learning Models Resistant to Adversarial Attackscited by 6,415
  4. 2022Robust Speech Recognition via Large-Scale Weak Supervisioncited by 4,057
  5. 2023Universal and Transferable Adversarial Attacks on Aligned Language Modelscited by 3,104
  6. 2016Towards Evaluating the Robustness of Neural Networkscited by 2,998
  7. 2023AutoDAN: Generating Stealthy Jailbreak Prompts on Aligned Large Language Modelscited by 785
  8. 2016Universal adversarial perturbationscited by 784
  9. 2018Speech Commands: A Dataset for Limited-Vocabulary Speech Recognitioncited by 752
  10. 2023Tree of Attacks: Jailbreaking Black-Box LLMs Automaticallycited by 672
  11. 2023Qwen-Audio: Advancing Universal Audio Understanding via Unified Large-Scale Audio-Language Modelscited by 608
  12. 2024Qwen2-Audio Technical Reportcited by 560
29 more
  1. 2023SpeechGPT: Empowering Large Language Models with Intrinsic Cross-Modal Conversational Abilitiescited by 529
  2. 2023SALMONN: Towards Generic Hearing Abilities for Large Language Modelscited by 506
  3. 2023AudioPaLM: A Large Language Model That Can Speak and Listencited by 343
  4. 2018Audio Adversarial Examples: Targeted Attacks on Speech-to-Textcited by 274
  5. 2023AudioGPT: Understanding and Generating Speech, Music, Sound, and Talking Headcited by 265
  6. 2023Pengi: An Audio Language Model for Audio Taskscited by 220
  7. 2023Open Sesame! Universal Black Box Jailbreaking of Large Language Modelscited by 134
  8. 2024Spirit LM: Interleaved Spoken and Written Language Modelcited by 126
  9. 2023SeamlessM4T: Massively Multilingual & Multimodal Machine Translationcited by 125
  10. 2019Imperceptible, Robust, and Targeted Adversarial Examples for Automatic Speech Recognitioncited by 124
  11. 2024Mini-Omni2: Towards Open-source GPT-4o with Vision, Speech and Duplex Capabilitiescited by 109
  12. 2025MinMo: A Multimodal Large Language Model for Seamless Voice Interactioncited by 79
  13. 2024Don't Say No: Jailbreaking LLM by Suppressing Refusalcited by 56
  14. 2025Playing the Fool: Jailbreaking LLMs and Multimodal LLMs with Out-of-Distribution Strategycited by 44
  15. 2024Voice Jailbreak Attacks Against GPT-4ocited by 42
  16. 2019Universal Adversarial Perturbations for Speech Recognition Systemscited by 41
  17. 2023Sparks of Large Audio Models: A Survey and Outlookcited by 41
  18. 2024AdvWave: Stealthy Adversarial Jailbreak Attack against Large Audio-Language Modelscited by 38
  19. 2021Multilingual and Cross-Lingual Intent Detection from Spoken Datacited by 35
  20. 2025Granite-speech: open-source speech-aware LLMs with strong English ASR capabilitiescited by 27
  21. 2023I See Dead People: Gray-Box Adversarial Attack on Image-To-Text Modelscited by 17
  22. 2022Foiling Explanations in Deep Neural Networkscited by 14
  23. 2020Universal Adversarial Perturbations Generative Network for Speaker Recognitioncited by 13
  24. 2019Universal Adversarial Audio Perturbationscited by 11
  25. 2022An Evolutionary, Gradient-Free, Query-Efficient, Black-Box Algorithm for Generating Adversarial Instances in Deep Networkscited by 11
  26. 2024On the Robustness of Kolmogorov-Arnold Networks: An Adversarial Perspectivecited by 11
  27. 2025Attacker's Noise Can Manipulate Your Audio-based LLM in the Real Worldcited by 9
  28. 2024Fortify the Guardian, Not the Treasure: Resilient Adversarial Detectorscited by 4
  29. 2025Backdoors in Conditional Diffusion: Threats to Responsible Synthetic Data Pipelinescited by 1

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2026: 4.

4 papers.

  1. 2026Codec-Robust Attacks on Audio LLMscited by 1
  2. 2026Audio Jailbreaks in Large Audio-Language Models: Taxonomy, Attack-Defense Analysis, and Cost-Aware Evaluationcited by 1
  3. 2026Exploiting Neural Audio Codec Latents for Adversarial Audio Attackscited by 0
  4. 2026From Acoustic Sensing to Physical Actions: A Comprehensive Survey of Security Risks in Edge-Deployed Audio LLM-Based Agentscited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2026Hijacking Large Audio-Language Models via Context-Agnostic and Imperceptible Auditory Prompt Injection
  2. 2025"I am bad": Interpreting Stealthy, Universal and Robust Audio Jailbreaks in Audio-Language Models
  3. 2025Multilingual and Multi-Accent Jailbreaking of Audio LLMs
  4. 2025When Good Sounds Go Adversarial: Jailbreaking Audio-Language Models with Benign Inputs
  5. 2025ALMGuard: Safety Shortcuts and Where to Find Them as Guardrails for Audio-Language Models
  6. 2026A Survey of Large Audio Language Models: Generalization, Trustworthiness, and Outlook
  7. 2025Audio Jailbreak Attacks: Exposing Vulnerabilities in SpeechGPT in a White-Box Framework
  8. 2025Investigating Vulnerabilities and Defenses Against Audio-Visual Attacks: A Comprehensive Survey Emphasizing Multimodal Models