mlsanityData to 14 Oct 2026

JALMBench: Benchmarking Jailbreak Vulnerabilities in Audio Language Models

arXiv 2505.17568 ↗First posted 23 May 2025cs.CR cs.AI cs.SDcited by 16

On the map: LLM safety and security → Audio jailbreaks and speech backdoors

Abstract (arXiv)

Large Audio Language Models (LALMs) have made significant progress. While increasingly deployed in real-world applications, LALMs face growing safety risks from jailbreak attacks that bypass safety alignment. However, there remains a lack of an adversarial audio dataset and a unified framework specifically designed to evaluate and compare jailbreak attacks against them. To address this gap, we introduce JALMBench, a comprehensive benchmark that assesses LALM safety against jailbreak attacks, comprising 11,316 text samples and 245,355 audio samples (>1,000 hours). JALMBench supports 12 mainstream LALMs, 8 attack methods (4 text-transferred and 4 audio-originated), and 5 defenses. We conduct in-depth analysis on attack efficiency, topic sensitivity, voice diversity, and model architecture. Additionally, we explore mitigation strategies for the attacks at both the prompt and response levels. Our systematic evaluation reveals that LALMs' safety is strongly influenced by modality and architectural choices: text-based safety alignment can partially transfer to audio inputs, and interleaved audio-text strategies enable more robust cross-modal generalization. Existing general-purpose moderation methods only slightly improve security, highlighting the need for defense methods specifically designed for LALMs. We hope our work can shed light on the design principles for building more robust LALMs.

Key facts. Cited by 16 papers in the map (data through 14 Oct 2026). Placed in Audio jailbreaks and speech backdoors, one of 12 directions in LLM safety and security. Builds on 47 of its 60 references that are in the map. First posted on arXiv 23 May 2025.

Builds on

47 of 60 references.

Closest ancestors · 4

Same direction, newest first

  1. 2025Audio Jailbreak: An Open Comprehensive Benchmark for Jailbreaking Large Audio-Language Models21 May
  2. 2024AdvWave: Stealthy Adversarial Jailbreak Attack against Large Audio-Language Models11 Dec
  3. 2024Audio Is the Achilles' Heel: Red Teaming Audio Large Multimodal Models31 Oct
  4. 2024Voice Jailbreak Attacks Against GPT-4o29 May

Other references · 43

  1. 2022Training language models to follow instructions with human feedbackcited by 15,971
  2. 2022Robust Speech Recognition via Large-Scale Weak Supervisioncited by 4,057
  3. 2023Universal and Transferable Adversarial Attacks on Aligned Language Modelscited by 3,104
  4. 2021HuBERT: Self-Supervised Speech Representation Learning by Masked Prediction of Hidden Unitscited by 2,342
  5. 2018Can a Suit of Armor Conduct Electricity? A New Dataset for Open Book Question Answeringcited by 2,164
  6. 2024HarmBench: A Standardized Evaluation Framework for Automated Red Teaming and Robust Refusalcited by 1,429
  7. 2023A Survey on Evaluation of Large Language Modelscited by 1,242
  8. 2023Llama Guard: LLM-based Input-Output Safeguard for Human-AI Conversationscited by 1,207
  9. 2023Fine-tuning Aligned Language Models Compromises Safety, Even When Users Do Not Intend To!cited by 1,154
  10. 2024How Johnny Can Persuade LLMs to Jailbreak Them: Rethinking Persuasion to Challenge AI Safety by Humanizing LLMscited by 573
  11. 2024Qwen2-Audio Technical Reportcited by 560
  12. 2024JailbreakBench: An Open Robustness Benchmark for Jailbreaking Large Language Modelscited by 536
31 more
  1. 2023SpeechGPT: Empowering Large Language Models with Intrinsic Cross-Modal Conversational Abilitiescited by 529
  2. 2023SALMONN: Towards Generic Hearing Abilities for Large Language Modelscited by 506
  3. 2023"Do Anything Now": Characterizing and Evaluating In-The-Wild Jailbreak Prompts on Large Language Modelscited by 489
  4. 2023SmoothLLM: Defending Large Language Models Against Jailbreaking Attackscited by 469
  5. 2023Detecting Language Model Attacks with Perplexitycited by 431
  6. 2023Jailbreak and Guard Aligned Language Models with Only Few In-Context Demonstrationscited by 431
  7. 2023Scaling Speech Technology to 1,000+ Languagescited by 411
  8. 2023FigStep: Jailbreaking Large Vision-Language Models via Typographic Visual Promptscited by 398
  9. 2023Safety-Tuned LLaMAs: Lessons From Improving the Safety of Large Language Models that Follow Instructionscited by 387
  10. 2023A Survey on Large Language Model (LLM) Security and Privacy: The Good, the Bad, and the Uglycited by 387
  11. 2023DeepInception: Hypnotize Large Language Model to Be Jailbreakercited by 367
  12. 2024F5-TTS: A Fairytaler that Fakes Fluent and Faithful Speech with Flow Matchingcited by 361
  13. 2023MM-SafetyBench: A Benchmark for Safety Evaluation of Multimodal Large Language Modelscited by 274
  14. 2024SafeDecoding: Defending against Jailbreak Attacks via Safety-Aware Decodingcited by 247
  15. 2024Jailbreak Attacks and Defenses Against Large Language Models: A Surveycited by 230
  16. 2024GLM-4-Voice: Towards Intelligent and Human-Like End-to-End Spoken Chatbotcited by 224
  17. 2025VITA-1.5: Towards GPT-4o Level Real-Time Vision and Speech Interactioncited by 197
  18. 2023MasterKey: Automated Jailbreak Across Multiple Large Language Model Chatbotscited by 183
  19. 2021SpeechT5: Unified-Modal Encoder-Decoder Pre-Training for Spoken Language Processingcited by 180
  20. 2024VITA: Towards Open-Source Interactive Omni Multimodal LLMcited by 173
  21. 2024LLaMA-Omni: Seamless Speech Interaction with Large Language Modelscited by 173
  22. 2024Freeze-Omni: A Smart and Low Latency Speech-to-speech Dialogue Model with Frozen LLMcited by 141
  23. 2024Spirit LM: Interleaved Spoken and Written Language Modelcited by 126
  24. 2024AdaShield: Safeguarding Multimodal Large Language Models from Structure-based Attack via Adaptive Shield Promptingcited by 125
  25. 2023Spoken Question Answering and Speech Continuation Using Spectrogram-Powered LLMcited by 94
  26. 2024Defending Large Language Models against Jailbreak Attacks via Semantic Smoothingcited by 74
  27. 2024A Survey on Speech Large Language Models for Understandingcited by 71
  28. 2023Improving Audio Captioning Models with Fine-grained Audio Features, Text Embedding Supervision, and LLM Mix-up Augmentationcited by 19
  29. 2024Quantized Delta Weight Is Safety Keepercited by 12
  30. 2024On the Evaluation of Speech Foundation Models for Spoken Language Understandingcited by 9
  31. 2024CL-Attack: Textual Backdoor Attacks via Cross-Lingual Triggerscited by 8

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2025: 3; 2026: 13.

16 papers.

Sort:

By area:

  1. 2025Speech-Audio Compositional Attacks on Multimodal LLMs and Their Mitigation with SALMONN-Guardcited by 13
  2. 2026VoxSafeBench: Not Just What Is Said, but Who, How, and Wherecited by 8
  3. 2025Investigating Safety Vulnerabilities of Large Audio-Language Models Under Speaker Emotional Variationscited by 4
  4. 2026Hijacking Large Audio-Language Models via Context-Agnostic and Imperceptible Auditory Prompt Injectioncited by 3
  5. 2026Acoustic Interference: A New Paradigm Weaponizing Acoustic Latent Semantic for Universal Jailbreak against Large Audio Language Modelscited by 3
  6. 2026"What Did It Actually Do?": Understanding Risk Awareness and Traceability for Computer-Use Agentscited by 2
  7. 2025Rebellion: Noise-Robust Reasoning Training for Audio Reasoning Modelscited by 1
  8. 2026The Alignment Curse: Modality Alignment Supercharges Audio Attacks via Text Transfercited by 1
  9. 2026On Optimizing Multimodal Jailbreaks for Spoken Language Modelscited by 1
  10. 2026Safety Geometry Collapse in Multimodal LLMs and Adaptive Drift Correctioncited by 1
  11. 2026Audio Jailbreaks in Large Audio-Language Models: Taxonomy, Attack-Defense Analysis, and Cost-Aware Evaluationcited by 1
  12. 2026AEGIS: Audio Endogenous Guarding via Internal Signals Against Large Audio-Language Model Jailbreakscited by 1
4 more
  1. 2026RedVox: Safety and Fairness Gaps in Speech Models Across Languagescited by 0
  2. 2026Causal Localization of the Refusal Direction in Audio Language Modelscited by 0
  3. 2026Psychoacoustically Aligned Latent Smoothing for Adversarial Robustness of Full-Duplex Speech-to-Speech Dialogue Modelscited by 0
  4. 2026From Acoustic Sensing to Physical Actions: A Comprehensive Survey of Security Risks in Edge-Deployed Audio LLM-Based Agentscited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2025Jailbreak-AudioBench: In-Depth Evaluation and Analysis of Jailbreak Threats for Large Audio Language Models
  2. 2025`From Prompt to Perturbation': An Adaptive Framework for Voice-Based Jailbreaks on Audio LLMs
  3. 2025AudioJailbreak: Jailbreak Attacks against End-to-End Large Audio-Language Models
  4. 2025Multilingual and Multi-Accent Jailbreaking of Audio LLMs
  5. 2026GRM: Utility-Aware Jailbreak Attacks on Audio LLMs via Gradient-Ratio Masking
  6. 2025"I am bad": Interpreting Stealthy, Universal and Robust Audio Jailbreaks in Audio-Language Models
  7. 2024JailBreakV: A Benchmark for Assessing the Robustness of MultiModal Large Language Models against Jailbreak Attacks
  8. 2024Divide and Conquer: A Hybrid Strategy Defeats Multimodal Large Language Models