# Rebellion: Noise-Robust Reasoning Training for Audio Reasoning Models

arXiv: https://arxiv.org/abs/2511.09682 · HTML: https://mlsanity.com/paper/2511.09682/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Placed in Audio-language model reasoning, one of 17 directions in LLM reasoning.
- Builds on 29 of its 33 references that are in the map.
- First posted on arXiv 12 Nov 2025.

## Placed in

- LLM reasoning / [Audio-language model reasoning](https://mlsanity.com/lr/audio-language-model-reasoning/)
- LLM safety and security / [Audio jailbreaks and speech backdoors](https://mlsanity.com/safety/audio-jailbreaks-and-speech-backdoors/)

## Builds on (most cited first, 29 of 29)

- [Training a Helpful and Harmless Assistant with Reinforcement Learning from Human Feedback](https://mlsanity.com/paper/2204.05862/) (2022)
- [Universal and Transferable Adversarial Attacks on Aligned Language Models](https://mlsanity.com/paper/2307.15043/) (2023)
- [Sharpness-Aware Minimization for Efficiently Improving Generalization](https://mlsanity.com/paper/2010.01412/) (2020)
- [Safe RLHF: Safe Reinforcement Learning from Human Feedback](https://mlsanity.com/paper/2310.12773/) (2023)
- [Qwen2-Audio Technical Report](https://mlsanity.com/paper/2407.10759/) (2024)
- [Deliberative Alignment: Reasoning Enables Safer Language Models](https://mlsanity.com/paper/2412.16339/) (2024)
- [Audio Flamingo 3: Advancing Audio Intelligence with Fully Open Large Audio Language Models](https://mlsanity.com/paper/2507.08128/) (2025)
- [SafeChain: Safety of Language Models with Long Chain-of-Thought Reasoning Capabilities](https://mlsanity.com/paper/2502.12025/) (2025)
- [Audio-Reasoner: Improving Reasoning Capability in Large Audio Language Models](https://mlsanity.com/paper/2503.02318/) (2025)
- [STAR-1: Safer Alignment of Reasoning LLMs with 1K Data](https://mlsanity.com/paper/2504.01903/) (2025)
- [Reinforcement Learning Outperforms Supervised Fine-Tuning: A Case Study on Audio Question Answering](https://mlsanity.com/paper/2503.11197/) (2025)
- [Omni-R1: Do You Really Need Audio to Fine-Tune Your Audio LLM?](https://mlsanity.com/paper/2505.09439/) (2025)
- [Voice Jailbreak Attacks Against GPT-4o](https://mlsanity.com/paper/2405.19103/) (2024)
- [Unveiling the Safety of GPT-4o: An Empirical Study using Jailbreak Attacks](https://mlsanity.com/paper/2406.06302/) (2024)
- [AdvWave: Stealthy Adversarial Jailbreak Attack against Large Audio-Language Models](https://mlsanity.com/paper/2412.08608/) (2024)
- [Audio Is the Achilles' Heel: Red Teaming Audio Large Multimodal Models](https://mlsanity.com/paper/2410.23861/) (2024)
- [SARI: Structured Audio Reasoning via Curriculum-Guided Reinforcement Learning](https://mlsanity.com/paper/2504.15900/) (2025)
- [Multilingual and Multi-Accent Jailbreaking of Audio LLMs](https://mlsanity.com/paper/2504.01094/) (2025)
- [Jailbreak-AudioBench: In-Depth Evaluation and Analysis of Jailbreak Threats for Large Audio Language Models](https://mlsanity.com/paper/2501.13772/) (2025)
- [Audio Jailbreak: An Open Comprehensive Benchmark for Jailbreaking Large Audio-Language Models](https://mlsanity.com/paper/2505.15406/) (2025)
- [SpeechGuard: Exploring the Adversarial Robustness of Multimodal Large Language Models](https://mlsanity.com/paper/2405.08317/) (2024)
- [AudioJailbreak: Jailbreak Attacks against End-to-End Large Audio-Language Models](https://mlsanity.com/paper/2505.14103/) (2025)
- [JALMBench: Benchmarking Jailbreak Vulnerabilities in Audio Language Models](https://mlsanity.com/paper/2505.17568/) (2025)
- [SoundMind: RL-Incentivized Logic Reasoning for Audio-Language Models](https://mlsanity.com/paper/2506.12935/) (2025)
- [Evaluating Robustness of Large Audio Language Models to Audio Injection: An Empirical Study](https://mlsanity.com/paper/2505.19598/) (2025)
- [SPIRIT: Patching Speech Language Models against Jailbreak Attacks](https://mlsanity.com/paper/2505.13541/) (2025)
- ["I am bad": Interpreting Stealthy, Universal and Robust Audio Jailbreaks in Audio-Language Models](https://mlsanity.com/paper/2502.00718/) (2025)
- [Audio Jailbreak Attacks: Exposing Vulnerabilities in SpeechGPT in a White-Box Framework](https://mlsanity.com/paper/2505.18864/) (2025)
- [When Good Sounds Go Adversarial: Jailbreaking Audio-Language Models with Benign Inputs](https://mlsanity.com/paper/2508.03365/) (2025)

## Built on by (most cited first, 1 of 1)

- [Mega-ASR: Towards In-the-wild^2 Speech Recognition via Scaling up Real-world Acoustic Simulation](https://mlsanity.com/paper/2605.19833/) (2026)

## Abstract (arXiv)

Instilling reasoning capabilities in large models (LMs) using reasoning training (RT) significantly improves LMs' performances. Thus Audio Reasoning Models (ARMs), i.e., audio LMs that can reason, are becoming increasingly popular. However, no work has studied the safety of ARMs against jailbreak attacks that aim to elicit harmful responses from target models. To this end, first, we show that standard RT with appropriate safety reasoning data can protect ARMs from vanilla audio jailbreaks, but cannot protect them against our proposed simple yet effective jailbreaks. We show that this is because of the significant representation drift between vanilla and advanced jailbreaks which forces the target ARMs to emit harmful responses. Based on this observation, we propose Rebellion, a robust RT that trains ARMs to be robust to the worst-case representation drift. All our results are on Qwen2-Audio; they demonstrate that Rebellion: 1) can protect against advanced audio jailbreaks without compromising performance on benign tasks, and 2) significantly improves accuracy-safety trade-off over standard RT method.