mlsanityData to 14 Oct 2026

Audio Is the Achilles' Heel: Red Teaming Audio Large Multimodal Models

arXiv 2410.23861 ↗First posted 31 Oct 2024cs.CL cs.MM cs.SDcited by 37

On the map: Multimodal LLMs / vision-language models → MLLM jailbreaks and adversarial robustness

also in 1 more
Abstract (arXiv)

Large Multimodal Models (LMMs) have demonstrated the ability to interact with humans under real-world conditions by combining Large Language Models (LLMs) and modality encoders to align multimodal information (visual and auditory) with text. However, such models raise new safety challenges of whether models that are safety-aligned on text also exhibit consistent safeguards for multimodal inputs. Despite recent safety-alignment research on vision LMMs, the safety of audio LMMs remains under-explored. In this work, we comprehensively red team the safety of five advanced audio LMMs under three settings: (i) harmful questions in both audio and text formats, (ii) harmful questions in text format accompanied by distracting non-speech audio, and (iii) speech-specific jailbreaks. Our results under these settings demonstrate that open-source audio LMMs suffer an average attack success rate of 69.14% on harmful audio questions, and exhibit safety vulnerabilities when distracted with non-speech audio noise. Our speech-specific jailbreaks on Gemini-1.5-Pro achieve an attack success rate of 70.67% on the harmful query benchmark. We provide insights on what could cause these reported safety-misalignments. Warning: this paper contains offensive examples.

Key facts. Cited by 37 papers in the map (data through 14 Oct 2026). Placed in MLLM jailbreaks and adversarial robustness, one of 16 directions in Multimodal LLMs / vision-language models. Builds on 22 of its 35 references that are in the map. First posted on arXiv 31 Oct 2024.

Builds on

22 of 35 references.

Closest ancestors · 4

Same direction, newest first

  1. 2024SPA-VL: A Comprehensive Safety Preference Alignment Dataset for Vision Language Model17 Jun
  2. 2024Jailbreak Vision Language Models via Bi-Modal Adversarial Prompt6 Jun
  3. 2023FigStep: Jailbreaking Large Vision-Language Models via Typographic Visual Prompts9 Nov
  4. 2023Jailbreak in pieces: Compositional Adversarial Attacks on Multi-Modal Language Models26 Jul

Other references · 18

  1. 2023GPT-4 Technical Reportcited by 18,396
  2. 2023LLaMA: Open and Efficient Foundation Language Modelscited by 14,132
  3. 2024Gemini 1.5: Unlocking multimodal understanding across millions of tokens of contextcited by 3,224
  4. 2024HarmBench: A Standardized Evaluation Framework for Automated Red Teaming and Robust Refusalcited by 1,429
  5. 2023Jailbreaking Black Box Large Language Models in Twenty Queriescited by 1,420
  6. 2023Llama Guard: LLM-based Input-Output Safeguard for Human-AI Conversationscited by 1,207
  7. 2023Qwen-Audio: Advancing Universal Audio Understanding via Unified Large-Scale Audio-Language Modelscited by 608
  8. 2024How Johnny Can Persuade LLMs to Jailbreak Them: Rethinking Persuasion to Challenge AI Safety by Humanizing LLMscited by 573
  9. 2023Safety-Tuned LLaMAs: Lessons From Improving the Safety of Large Language Models that Follow Instructionscited by 387
  10. 2023DeepInception: Hypnotize Large Language Model to Be Jailbreakercited by 367
  11. 2023Do-Not-Answer: A Dataset for Evaluating Safeguards in LLMscited by 184
  12. 2024ALERT: A Comprehensive Benchmark for Assessing Large Language Models' Safety through Red Teamingcited by 98
6 more
  1. 2023CValues: Measuring the Values of Chinese Large Language Models from Safety to Responsibilitycited by 85
  2. 2023Cognitive Overload: Jailbreaking Large Language Models with Overloaded Logical Thinkingcited by 81
  3. 2024SafetyPrompts: a Systematic Review of Open Datasets for Evaluating and Improving Large Language Model Safetycited by 77
  4. 2023Self-Guard: Empower the LLM to Safeguard Itselfcited by 54
  5. 2023Walking a Tightrope -- Evaluating Large Language Models in High-Risk Domainscited by 15
  6. 2023Investigating Pre-trained Audio Encoders in the Low-Resource Conditioncited by 11

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2024: 2; 2025: 17; 2026: 18.

37 papers.

Sort:

By area:

  1. 2024Best-of-N Jailbreakingcited by 76
  2. 2024Jailbreak Attacks and Defenses against Multimodal Generative Models: A Surveycited by 32
  3. 2025Multilingual and Multi-Accent Jailbreaking of Audio LLMscited by 31
  4. 2025Jailbreak-AudioBench: In-Depth Evaluation and Analysis of Jailbreak Threats for Large Audio Language Modelscited by 23
  5. 2025Audio Jailbreak: An Open Comprehensive Benchmark for Jailbreaking Large Audio-Language Modelscited by 21
  6. 2025Omni-SafetyBench: A Benchmark for Safety Evaluation of Audio-Visual Large Language Modelscited by 18
  7. 2025JALMBench: Benchmarking Jailbreak Vulnerabilities in Audio Language Modelscited by 16
  8. 2025Speech-Audio Compositional Attacks on Multimodal LLMs and Their Mitigation with SALMONN-Guardcited by 13
  9. 2025"I am bad": Interpreting Stealthy, Universal and Robust Audio Jailbreaks in Audio-Language Modelscited by 8
  10. 2025SARSteer: Safeguarding Large Audio-Language Models via Safe-Ablated Refusal Steeringcited by 8
  11. 2025Reshaping Representation Space to Balance the Safety and Over-rejection in Large Audio Language Modelscited by 7
  12. 2025Universal Acoustic Adversarial Attacks for Flexible Control of Speech-LLMscited by 6
25 more
  1. 2025SAKE: Towards Editing Auditory Attribute Knowledge of Large Audio-Language Modelscited by 5
  2. 2025Investigating Safety Vulnerabilities of Large Audio-Language Models Under Speaker Emotional Variationscited by 4
  3. 2026Adversarial Attacks on Multimodal Large Language Models: A Comprehensive Surveycited by 4
  4. 2026Hijacking Large Audio-Language Models via Context-Agnostic and Imperceptible Auditory Prompt Injectioncited by 3
  5. 2026Acoustic Interference: A New Paradigm Weaponizing Acoustic Latent Semantic for Universal Jailbreak against Large Audio Language Modelscited by 3
  6. 2025Watch, Listen, Understand, Mislead: Tri-modal Adversarial Attacks on Short Videos for Content Appropriateness Evaluationcited by 2
  7. 2025ERIS: Evolutionary Real-world Interference Scheme for Jailbreaking Audio Large Modelscited by 2
  8. 2026MUGEN: Evaluating and Improving Multi-audio Understanding of Large Audio-Language Modelscited by 2
  9. 2026Resurfacing Paralinguistic Awareness in Large Audio Language Modelscited by 2
  10. 2026CASA: Classification Augmented with Safety Attention for Robust Multimodal Alignmentcited by 2
  11. 2026GRM: Utility-Aware Jailbreak Attacks on Audio LLMs via Gradient-Ratio Maskingcited by 2
  12. 2026Sparse Tokens Suffice: Jailbreaking Audio Language Models via Token-Aware Gradient Optimizationcited by 2
  13. 2025Investigating Vulnerabilities and Defenses Against Audio-Visual Attacks: A Comprehensive Survey Emphasizing Multimodal Modelscited by 1
  14. 2025Beyond Text: Multimodal Jailbreaking of Vision-Language and Audio Models through Perceptually Simple Transformationscited by 1
  15. 2025Rebellion: Noise-Robust Reasoning Training for Audio Reasoning Modelscited by 1
  16. 2026The Alignment Curse: Modality Alignment Supercharges Audio Attacks via Text Transfercited by 1
  17. 2026Aegis: Towards Governance, Integrity, and Security of AI Voice Agentscited by 1
  18. 2026Evaluation of Audio Language Models for Fairness, Safety, and Securitycited by 1
  19. 2026On Optimizing Multimodal Jailbreaks for Spoken Language Modelscited by 1
  20. 2026Benign Fine-Tuning Breaks Safety Alignment in Audio LLMscited by 1
  21. 2026AEGIS: Audio Endogenous Guarding via Internal Signals Against Large Audio-Language Model Jailbreakscited by 1
  22. 2026The Persistent Vulnerability of Aligned AI Systemscited by 0
  23. 2026DuplexJail: Safety Alignment Breaks Under Spoken Interruption in Full-Duplex Modelscited by 0
  24. 2026Causal Localization of the Refusal Direction in Audio Language Modelscited by 0
  25. 2026Where Does the Audio Jailbreak Live? A Controlled Frequency-Depth Audit of AdvWave-P on Qwen2-Audiocited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2026Now You Hear Me: Audio Narrative Attacks Against Large Audio-Language Models
  2. 2026AudioGuard: Toward Comprehensive Audio Safety Protection Across Diverse Threat Models
  3. 2026ARENA: Automated Red-Teaming for Large Audio Language Models
  4. 2026SpeechJBB: Probing Safety Alignment and Comprehension in Large Audio Language Models under Code-Switched Speech
  5. 2025ALMGuard: Safety Shortcuts and Where to Find Them as Guardrails for Audio-Language Models
  6. 2026A Survey of Large Audio Language Models: Generalization, Trustworthiness, and Outlook
  7. 2025Can't See the Forest for the Trees: Benchmarking Multimodal Safety Awareness for Multimodal LLMs
  8. 2025LLaVAShield: Safeguarding Multimodal Multi-Turn Dialogues in Vision-Language Models