# When Good Sounds Go Adversarial: Jailbreaking Audio-Language Models with Benign Inputs

arXiv: https://arxiv.org/abs/2508.03365 · HTML: https://mlsanity.com/paper/2508.03365/

## Key facts

- Cited by 4 papers in the map (data through 14 Oct 2026).
- Placed in Audio jailbreaks and speech backdoors, one of 12 directions in LLM safety and security.
- Builds on 27 of its 41 references that are in the map.
- First posted on arXiv 5 Aug 2025.

## Placed in

- LLM safety and security / [Audio jailbreaks and speech backdoors](https://mlsanity.com/safety/audio-jailbreaks-and-speech-backdoors/)

## Builds on (most cited first, 27 of 27)

- [Language Models are Few-Shot Learners](https://mlsanity.com/paper/2005.14165/) (2020)
- [The Llama 3 Herd of Models](https://mlsanity.com/paper/2407.21783/) (2024)
- [Explaining and Harnessing Adversarial Examples](https://mlsanity.com/paper/1412.6572/) (2014)
- [Towards Deep Learning Models Resistant to Adversarial Attacks](https://mlsanity.com/paper/1706.06083/) (2017)
- [Intriguing properties of neural networks](https://mlsanity.com/paper/1312.6199/) (2013)
- [wav2vec 2.0: A Framework for Self-Supervised Learning of Speech Representations](https://mlsanity.com/paper/2006.11477/) (2020)
- [Universal and Transferable Adversarial Attacks on Aligned Language Models](https://mlsanity.com/paper/2307.15043/) (2023)
- [Towards Evaluating the Robustness of Neural Networks](https://mlsanity.com/paper/1608.04644/) (2016)
- [Adversarial examples in the physical world](https://mlsanity.com/paper/1607.02533/) (2016)
- [Qwen2.5-Omni Technical Report](https://mlsanity.com/paper/2503.20215/) (2025)
- [DetectGPT: Zero-Shot Machine-Generated Text Detection using Probability Curvature](https://mlsanity.com/paper/2301.11305/) (2023)
- [JailbreakBench: An Open Robustness Benchmark for Jailbreaking Large Language Models](https://mlsanity.com/paper/2404.01318/) (2024)
- [Phi-4-Mini Technical Report: Compact yet Powerful Multimodal Language Models via Mixture-of-LoRAs](https://mlsanity.com/paper/2503.01743/) (2025)
- [A StrongREJECT for Empty Jailbreaks](https://mlsanity.com/paper/2402.10260/) (2024)
- [Audio Adversarial Examples: Targeted Attacks on Speech-to-Text](https://mlsanity.com/paper/1801.01944/) (2018)
- [AmpleGCG: Learning a Universal and Transferable Generative Model of Adversarial Suffixes for Jailbreaking Both Open and Closed LLMs](https://mlsanity.com/paper/2404.07921/) (2024)
- [Imperceptible, Robust, and Targeted Adversarial Examples for Automatic Speech Recognition](https://mlsanity.com/paper/1903.10346/) (2019)
- [GradSafe: Detecting Jailbreak Prompts for LLMs via Safety-Critical Gradient Analysis](https://mlsanity.com/paper/2402.13494/) (2024)
- [Fast Adversarial Attacks on Language Models In One GPU Minute](https://mlsanity.com/paper/2402.15570/) (2024)
- [Universal Adversarial Perturbations for Speech Recognition Systems](https://mlsanity.com/paper/1905.03828/) (2019)
- [Boosting Jailbreak Attack with Momentum](https://mlsanity.com/paper/2405.01229/) (2024)
- [AdvWave: Stealthy Adversarial Jailbreak Attack against Large Audio-Language Models](https://mlsanity.com/paper/2412.08608/) (2024)
- [AttnGCG: Enhancing Jailbreaking Attacks on LLMs with Attention Manipulation](https://mlsanity.com/paper/2410.09040/) (2024)
- [JailbreakEval: An Integrated Toolkit for Evaluating Jailbreak Attempts Against Large Language Models](https://mlsanity.com/paper/2406.09321/) (2024)
- [Audio Jailbreak: An Open Comprehensive Benchmark for Jailbreaking Large Audio-Language Models](https://mlsanity.com/paper/2505.15406/) (2025)
- [Recent improvements of ASR models in the face of adversarial attacks](https://mlsanity.com/paper/2203.16536/) (2022)
- [Attacker's Noise Can Manipulate Your Audio-based LLM in the Real World](https://mlsanity.com/paper/2507.06256/) (2025)

## Built on by (most cited first, 4 of 4)

- [Rebellion: Noise-Robust Reasoning Training for Audio Reasoning Models](https://mlsanity.com/paper/2511.09682/) (2025)
- [Codec-Robust Attacks on Audio LLMs](https://mlsanity.com/paper/2605.20519/) (2026)
- [Audio Jailbreaks in Large Audio-Language Models: Taxonomy, Attack-Defense Analysis, and Cost-Aware Evaluation](https://mlsanity.com/paper/2605.30031/) (2026)
- [From Acoustic Sensing to Physical Actions: A Comprehensive Survey of Security Risks in Edge-Deployed Audio LLM-Based Agents](https://mlsanity.com/paper/2610.15390/) (2026)

## Abstract (arXiv)

As large language models (LLMs) become increasingly integrated into daily life, audio has emerged as a key interface for human-AI interaction. However, this convenience also introduces new vulnerabilities, making audio a potential attack surface for adversaries. Our research introduces WhisperInject, a two-stage adversarial audio attack framework that manipulates state-of-the-art audio language models to generate harmful content. Our method embeds harmful payloads as subtle perturbations into audio inputs that remain intelligible to human listeners. The first stage uses a novel reward-based white-box optimization method, Reinforcement Learning with Projected Gradient Descent (RL-PGD), to jailbreak the target model and elicit harmful native responses. This native harmful response then serves as the target for Stage 2, Payload Injection, where we use gradient-based optimization to embed subtle perturbations into benign audio carriers, such as weather queries or greeting messages. Our method achieves average attack success rates of 60-78% across two benchmarks and five multimodal LLMs, validated by multiple evaluation frameworks. Our work demonstrates a new class of practical, audio-native threats, moving beyond theoretical exploits to reveal a feasible and covert method for manipulating multimodal AI systems.