# Who Can Withstand Chat-Audio Attacks? An Evaluation Benchmark for Large Audio-Language Models

arXiv: https://arxiv.org/abs/2411.14842 · HTML: https://mlsanity.com/paper/2411.14842/

## Key facts

- Cited by 7 papers in the map (data through 14 Oct 2026).
- Builds on 26 of its 39 references that are in the map.
- First posted on arXiv 22 Nov 2024.

## Builds on (most cited first, 26 of 26)

- [GPT-4 Technical Report](https://mlsanity.com/paper/2303.08774/) (2023)
- [Explaining and Harnessing Adversarial Examples](https://mlsanity.com/paper/1412.6572/) (2014)
- [Intriguing properties of neural networks](https://mlsanity.com/paper/1312.6199/) (2013)
- [Robust Speech Recognition via Large-Scale Weak Supervision](https://mlsanity.com/paper/2212.04356/) (2022)
- [Gemini 1.5: Unlocking multimodal understanding across millions of tokens of context](https://mlsanity.com/paper/2403.05530/) (2024)
- [Common Voice: A Massively-Multilingual Speech Corpus](https://mlsanity.com/paper/1912.06670/) (2019)
- [EDA: Easy Data Augmentation Techniques for Boosting Performance on Text Classification Tasks](https://mlsanity.com/paper/1901.11196/) (2019)
- [OpenAssistant Conversations -- Democratizing Large Language Model Alignment](https://mlsanity.com/paper/2304.07327/) (2023)
- [Is BERT Really Robust? A Strong Baseline for Natural Language Attack on Text Classification and Entailment](https://mlsanity.com/paper/1907.11932/) (2019)
- [MELD: A Multimodal Multi-Party Dataset for Emotion Recognition in Conversations](https://mlsanity.com/paper/1810.02508/) (2018)
- [SpeechGPT: Empowering Large Language Models with Intrinsic Cross-Modal Conversational Abilities](https://mlsanity.com/paper/2305.11000/) (2023)
- [BERT-ATTACK: Adversarial Attack Against BERT Using BERT](https://mlsanity.com/paper/2004.09984/) (2020)
- [TVQA: Localized, Compositional Video Question Answering](https://mlsanity.com/paper/1809.01696/) (2018)
- [Generative Spoken Language Modeling from Raw Audio](https://mlsanity.com/paper/2102.01192/) (2021)
- [Audio Adversarial Examples: Targeted Attacks on Speech-to-Text](https://mlsanity.com/paper/1801.01944/) (2018)
- [Survey of Vulnerabilities in Large Language Models Revealed by Adversarial Attacks](https://mlsanity.com/paper/2310.10844/) (2023)
- [On decoder-only architecture for speech-to-text and large language model integration](https://mlsanity.com/paper/2307.03917/) (2023)
- [Prompting Large Language Models with Speech Recognition Abilities](https://mlsanity.com/paper/2307.11795/) (2023)
- [X-LLM: Bootstrapping Advanced Large Language Models by Treating Multi-Modalities as Foreign Languages](https://mlsanity.com/paper/2305.04160/) (2023)
- [Large Language Models as Zero-Shot Conversational Recommenders](https://mlsanity.com/paper/2308.10053/) (2023)
- [Universal Adversarial Perturbations for Speech Recognition Systems](https://mlsanity.com/paper/1905.03828/) (2019)
- [Enabling Fast and Universal Audio Adversarial Attack Using Generative Model](https://mlsanity.com/paper/2004.12261/) (2020)
- [Crafting Adversarial Examples For Speech Paralinguistics Applications](https://mlsanity.com/paper/1711.03280/) (2017)
- [Real-time, Universal, and Robust Adversarial Attacks Against Speaker Recognition Systems](https://mlsanity.com/paper/2003.02301/) (2020)
- [Deep Learning and Music Adversaries](https://mlsanity.com/paper/1507.04761/) (2015)
- [Practical Attacks on Voice Spoofing Countermeasures](https://mlsanity.com/paper/2107.14642/) (2021)

## Built on by (most cited first, 7 of 7)

- [Advances and Challenges in Foundation Agents: From Brain-Inspired Intelligence to Evolutionary, Collaborative, and Safe Systems](https://mlsanity.com/paper/2504.01990/) (2025)
- [SpeechR: A Benchmark for Speech Reasoning in Large Audio-Language Models](https://mlsanity.com/paper/2508.02018/) (2025)
- [Temporal Contrastive Decoding: A Training-Free Method for Large Audio-Language Models](https://mlsanity.com/paper/2604.15383/) (2026)
- [Protecting Bystander Privacy via Selective Hearing in Audio LLMs](https://mlsanity.com/paper/2512.06380/) (2025)
- [Beyond In-Domain Detection: SpikeScore for Cross-Domain Hallucination Detection](https://mlsanity.com/paper/2601.19245/) (2026)
- [Audio Jailbreaks in Large Audio-Language Models: Taxonomy, Attack-Defense Analysis, and Cost-Aware Evaluation](https://mlsanity.com/paper/2605.30031/) (2026)
- [AEGIS: Audio Endogenous Guarding via Internal Signals Against Large Audio-Language Model Jailbreaks](https://mlsanity.com/paper/2609.29287/) (2026)

## Abstract (arXiv)

Adversarial audio attacks pose a significant threat to the growing use of large audio-language models (LALMs) in voice-based human-machine interactions. While existing research focused on model-specific adversarial methods, real-world applications demand a more generalizable and universal approach to audio adversarial attacks. In this paper, we introduce the Chat-Audio Attacks (CAA) benchmark including four distinct types of audio attacks, which aims to explore the vulnerabilities of LALMs to these audio attacks in conversational scenarios. To evaluate the robustness of LALMs, we propose three evaluation strategies: Standard Evaluation, utilizing traditional metrics to quantify model performance under attacks; GPT-4o-Based Evaluation, which simulates real-world conversational complexities; and Human Evaluation, offering insights into user perception and trust. We evaluate six state-of-the-art LALMs with voice interaction capabilities, including Gemini-1.5-Pro, GPT-4o, and others, using three distinct evaluation methods on the CAA benchmark. Our comprehensive analysis reveals the impact of four types of audio attacks on the performance of these models, demonstrating that GPT-4o exhibits the highest level of resilience. Our data can be accessed via the following link: hrefhttps://github.com/crystraldo/CAACAA.