mlsanityData to 7 Oct 2026

SpeechGuard: Exploring the Adversarial Robustness of Multimodal Large Language Models

arXiv 2405.08317 ↗First posted 14 May 2024cs.CL cs.SD eess.AScited by 16

On the map: Multimodal LLMs / vision-language models → MLLM jailbreaks and adversarial robustness

also in 1 more
Abstract (arXiv)

Integrated Speech and Large Language Models (SLMs) that can follow speech instructions and generate relevant text responses have gained popularity lately. However, the safety and robustness of these models remains largely unclear. In this work, we investigate the potential vulnerabilities of such instruction-following speech-language models to adversarial attacks and jailbreaking. Specifically, we design algorithms that can generate adversarial examples to jailbreak SLMs in both white-box and black-box attack settings without human involvement. Additionally, we propose countermeasures to thwart such jailbreaking attacks. Our models, trained on dialog data with speech instructions, achieve state-of-the-art performance on spoken question-answering task, scoring over 80% on both safety and helpfulness metrics. Despite safety guardrails, experiments on jailbreaking demonstrate the vulnerability of SLMs to adversarial perturbations and transfer attacks, with average attack success rates of 90% and 10% respectively when evaluated on a dataset of carefully designed harmful questions spanning 12 different toxic categories. However, we demonstrate that our proposed countermeasures reduce the attack success significantly.

Key facts. Cited by 16 papers in the map (data through 7 Oct 2026). Placed in MLLM jailbreaks and adversarial robustness, one of 16 directions in Multimodal LLMs / vision-language models. Builds on 36 of its 58 references that are in the map. First posted on arXiv 14 May 2024.

Builds on

36 of 58 references.

Closest ancestors · 2

Same direction, newest first

  1. 2023How Robust is Google's Bard to Adversarial Image Attacks?21 Sep
  2. 2023Abusing Images and Sounds for Indirect Instruction Injection in Multi-Modal LLMs19 Jul

Other references · 34

  1. 2019PyTorch: An Imperative Style, High-Performance Deep Learning Librarycited by 20,160
  2. 2022Training language models to follow instructions with human feedbackcited by 15,712
  3. 2021LoRA: Low-Rank Adaptation of Large Language Modelscited by 14,084
  4. 2014Explaining and Harnessing Adversarial Examplescited by 7,287
  5. 2017Towards Deep Learning Models Resistant to Adversarial Attackscited by 6,326
  6. 2021On the Opportunities and Risks of Foundation Modelscited by 3,961
  7. 2022Constitutional AI: Harmlessness from AI Feedbackcited by 2,950
  8. 2023Universal and Transferable Adversarial Attacks on Aligned Language Modelscited by 2,914
  9. 2020Conformer: Convolution-augmented Transformer for Speech Recognitioncited by 1,838
  10. 2023Jailbroken: How Does LLM Safety Training Fail?cited by 1,565
  11. 2022Rethinking the Role of Demonstrations: What Makes In-Context Learning Work?cited by 1,413
  12. 2023Jailbreaking Black Box Large Language Models in Twenty Queriescited by 1,326
22 more
  1. 2019Certified Adversarial Robustness via Randomized Smoothingcited by 1,324
  2. 2021A General Language Assistant as a Laboratory for Alignmentcited by 1,023
  3. 2020Aligning AI With Shared Human Valuescited by 706
  4. 2023Baseline Defenses for Adversarial Attacks Against Aligned Language Modelscited by 608
  5. 2023SpeechGPT: Empowering Large Language Models with Intrinsic Cross-Modal Conversational Abilitiescited by 518
  6. 2018Black-box Adversarial Attacks with Limited Queries and Informationcited by 499
  7. 2023PandaLM: An Automatic Evaluation Benchmark for LLM Instruction Tuning Optimizationcited by 311
  8. 2023Certifying LLM Safety against Adversarial Promptingcited by 248
  9. 2023Pengi: An Audio Language Model for Audio Taskscited by 217
  10. 2023MART: Improving LLM Safety with Multi-round Automatic Red-Teamingcited by 161
  11. 2022Self-supervised Learning with Random-projection Quantizer for Speech Recognitioncited by 158
  12. 2024Continual Learning for Large Language Models: A Surveycited by 144
  13. 2023Text-to-Image Diffusion Models can be Easily Backdoored through Multimodal Data Poisoningcited by 91
  14. 2018Adversarial Attacks Against Automatic Speech Recognition Systems via Psychoacoustic Hidingcited by 83
  15. 2023SLM: Bridge the thin gap between speech and text foundation modelscited by 67
  16. 2023LLaSM: Large Language and Speech Modelcited by 56
  17. 2024Gradient-Based Language Model Red Teamingcited by 33
  18. 2023Open-Source LLMs for Text Annotation: A Practical Guide for Model Setting and Fine-Tuningcited by 31
  19. 2020Adversarial Attack and Defense Strategies for Deep Speaker Recognition Systemscited by 21
  20. 2023Summon a Demon and Bind it: A Grounded Theory of LLM Red Teamingcited by 20
  21. 2018Noise Flooding for Detecting Audio Adversarial Examples Against Automatic Speech Recognitioncited by 10
  22. 2023JAB: Joint Adversarial Prompting and Belief Augmentationcited by 5

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2024: 1; 2025: 8; 2026: 7.

16 papers.

Sort:

By area:

  1. 2025Advances and Challenges in Foundation Agents: From Brain-Inspired Intelligence to Evolutionary, Collaborative, and Safe Systemscited by 125
  2. 2025OverThink: Slowdown Attacks on Reasoning LLMscited by 94
  3. 2025AudioJailbreak: Jailbreak Attacks against End-to-End Large Audio-Language Modelscited by 16
  4. 2025Survey of Adversarial Robustness in Multimodal Large Language Modelscited by 14
  5. 2025`From Prompt to Perturbation': An Adaptive Framework for Voice-Based Jailbreaks on Audio LLMscited by 6
  6. 2025When Silence Matters: The Impact of Irrelevant Audio on Text Reasoning in Large Audio-Language Modelscited by 4
  7. 2025SmoothGuard: Defending Multimodal Large Language Models with Noise Perturbation and Clustering Aggregationcited by 2
  8. 2026AudioGuard: Toward Comprehensive Audio Safety Protection Across Diverse Threat Modelscited by 2
  9. 2026Hijacking Large Audio-Language Models via Context-Agnostic and Imperceptible Auditory Prompt Injectioncited by 2
  10. 2025Rebellion: Noise-Robust Reasoning Training for Audio Reasoning Modelscited by 1
  11. 2026Resurfacing Paralinguistic Awareness in Large Audio Language Modelscited by 1
  12. 2026Sparse Tokens Suffice: Jailbreaking Audio Language Models via Token-Aware Gradient Optimizationcited by 1
4 more
  1. 2024Consultation on Industrial Machine Faults with Large language Modelscited by 0
  2. 2026On Optimizing Multimodal Jailbreaks for Spoken Language Modelscited by 0
  3. 2026Codec-Robust Attacks on Audio LLMscited by 0
  4. 2026Psychoacoustically Aligned Latent Smoothing for Adversarial Robustness of Full-Duplex Speech-to-Speech Dialogue Modelscited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2025Audio Jailbreak Attacks: Exposing Vulnerabilities in SpeechGPT in a White-Box Framework
  2. 2026Adversarial Attacks on Multimodal Large Language Models: A Comprehensive Survey
  3. 2024Jailbreak Attacks and Defenses Against Large Language Models: A Survey
  4. 2025"I am bad": Interpreting Stealthy, Universal and Robust Audio Jailbreaks in Audio-Language Models
  5. 2023Survey of Vulnerabilities in Large Language Models Revealed by Adversarial Attacks
  6. 2025Multilingual and Multi-Accent Jailbreaking of Audio LLMs
  7. 2024Defending Large Language Models Against Jailbreak Attacks via Layer-specific Editing
  8. 2025Con Instruction: Universal Jailbreaking of Multimodal Large Language Models via Non-Textual Modalities