Investigating Vulnerabilities and Defenses Against Audio-Visual Attacks: A Comprehensive Survey Emphasizing Multimodal Models
On the map: Multimodal LLMs / vision-language models → MLLM jailbreaks and adversarial robustness
also in 1 more
Abstract (arXiv)
Multimodal large language models (MLLMs), which bridge the gap between audio-visual and natural language processing, achieve state-of-the-art performance on several audio-visual tasks. Despite the superior performance of MLLMs, the scarcity of high-quality audio-visual training data and computational resources necessitates the utilization of third-party data and open-source MLLMs, a trend that is increasingly observed in contemporary research. This prosperity masks significant security risks. Empirical studies demonstrate that the latest MLLMs can be manipulated to produce malicious or harmful content. This manipulation is facilitated exclusively through instructions or inputs, including adversarial perturbations and malevolent queries, effectively bypassing the internal security mechanisms embedded within the models. To gain a deeper comprehension of the inherent security vulnerabilities associated with audio-visual-based multimodal models, a series of surveys investigates various types of attacks, including adversarial and backdoor attacks. While existing surveys on audio-visual attacks provide a comprehensive overview, they are limited to specific types of attacks, which lack a unified review of various types of attacks. To address this issue and gain insights into the latest trends in the field, this paper presents a comprehensive and systematic review of audio-visual attacks, which include adversarial attacks, backdoor attacks, and jailbreak attacks. Furthermore, this paper also reviews various types of attacks in the latest audio-visual-based MLLMs, a dimension notably absent in existing surveys. Drawing upon comprehensive insights from a substantial review, this paper delineates both challenges and emergent trends for future research on audio-visual attacks and defense.
Key facts. Cited by 1 paper in the map (data through 13 Oct 2026). Placed in MLLM jailbreaks and adversarial robustness, one of 16 directions in Multimodal LLMs / vision-language models. Builds on 66 of its 134 references that are in the map. First posted on arXiv 13 Jun 2025.
Builds on
66 of 134 references.
Closest ancestors · 2
Same direction, newest first
- 2025Transferability of Adversarial Attacks in Video-based MLLMs: A Cross-modal Image-to-Video Approach2 Jan
- 2024Audio Is the Achilles' Heel: Red Teaming Audio Large Multimodal Models31 Oct
Other references · 64
- 2024GPT-4o System Cardcited by 4,536
- 2023Qwen-Audio: Advancing Universal Audio Understanding via Unified Large-Scale Audio-Language Modelscited by 601
- 2018Audio Adversarial Examples: Targeted Attacks on Speech-to-Textcited by 272
- 2024LLaMA-Omni: Seamless Speech Interaction with Large Language Modelscited by 172
- 2024A Comprehensive Study of Jailbreak Attack versus Defense for Large Language Modelscited by 106
- 2024JailbreakRadar: Comprehensive Assessment of Jailbreak Attacks Against LLMscited by 95
- 2023Prompt as Triggers for Backdoor Attack: Examining the Vulnerability in Language Modelscited by 93
- 2024Best-of-N Jailbreakingcited by 75
- 2024T2VSafetyBench: Evaluating the Safety of Text-to-Video Generative Modelscited by 66
- 2024Universal Vulnerabilities in Large Language Models: Backdoor Attacks for In-context Learningcited by 53
- 2024Automatic Speech Recognition using Advanced Deep Learning Approaches: A surveycited by 50
- 2021Can You Hear It? Backdoor Attacks via Ultrasonic Triggerscited by 44
38 more
- 2018Targeted Adversarial Examples for Black Box Audio Systemscited by 41
- 2024Voice Jailbreak Attacks Against GPT-4ocited by 40
- 2021WaveGuard: Understanding and Mitigating Audio Adversarial Examplescited by 39
- 2024Unveiling the Safety of GPT-4o: An Empirical Study using Jailbreak Attackscited by 38
- 2024AdvWave: Stealthy Adversarial Jailbreak Attack against Large Audio-Language Modelscited by 37
- 2020Enabling Fast and Universal Audio Adversarial Attack Using Generative Modelcited by 32
- 2024Jailbreak Attacks and Defenses against Multimodal Generative Models: A Surveycited by 32
- 2024Unlearning Backdoor Attacks for LLMs with Weak-to-Strong Knowledge Distillationcited by 26
- 2021Black-box Adversarial Attacks on Commercial Speech Platforms with Minimal Informationcited by 25
- 2022Towards Understanding and Mitigating Audio Adversarial Examples for Speaker Recognitioncited by 23
- 2023Defending against Adversarial Audio via Diffusion Modelcited by 21
- 2019Adversarial Music: Real World Audio Adversary Against Wake-word Detection Systemcited by 20
- 2022Going In Style: Audio Backdoors Through Stylistic Transformationscited by 20
- 2024SafeEar: Content Privacy-Preserving Audio Deepfake Detectioncited by 20
- 2025Seeing Sound, Hearing Sight: Uncovering Modality Bias and Conflict of AI models in Sound Localizationcited by 18
- 2024Defending Against Weight-Poisoning Backdoor Attacks for Parameter-Efficient Fine-Tuningcited by 16
- 2023ZMM-TTS: Zero-shot Multilingual and Multispeaker Speech Synthesis Conditioned on Self-supervised Discrete Speech Representationscited by 15
- 2024FMM-Attack: A Flow-based Multi-modal Adversarial Attack on Video-based LLMscited by 14
- 2024LlamaPartialSpoof: An LLM-Driven Fake Speech Dataset Simulating Disinformation Generationcited by 14
- 2025Uni-Retrieval: A Multi-Style Retrieval Framework for STEM's Educationcited by 13
- 2019A Robust Approach for Securing Audio Classification Against Adversarial Attackscited by 12
- 2024Adversarial Attacks to Multi-Modal Modelscited by 12
- 2024TrustNavGPT: Modeling Uncertainty to Improve Trustworthiness of Audio-Guided LLM-Based Robot Navigationcited by 11
- 2023FlowMur: A Stealthy and Practical Audio Backdoor Attack with Limited Knowledgecited by 10
- 2024Zero-Query Adversarial Attack on Black-box Automatic Speech Recognition Systemscited by 10
- 2025Transferable Adversarial Attacks on Audio Deepfake Detectioncited by 9
- 2019Detecting Adversarial Attacks On Audiovisual Speech Recognitioncited by 8
- 2023MASTERKEY: Practical Backdoor Attack Against Speaker Verification Systemscited by 8
- 2025"I am bad": Interpreting Stealthy, Universal and Robust Audio Jailbreaks in Audio-Language Modelscited by 8
- 2019Adversarial Audio: A New Information Hiding Method and Backdoor for DNN-based Speech Recognition Modelscited by 7
- 2023Look, Listen, and Attack: Backdoor Attacks Against Video Action Recognitioncited by 7
- 2023TransAudio: Towards the Transferable Adversarial Audio Attack via Learning Contextualized Perturbationscited by 7
- 2025`From Prompt to Perturbation': An Adaptive Framework for Voice-Based Jailbreaks on Audio LLMscited by 7
- 2025Towards Robust Evaluation of STEM Education: Leveraging MLLMs in Project-Based Learningcited by 7
- 2022Adversarial attacks and defenses in Speaker Recognition Systems: A surveycited by 6
- 2023Backdoor Attacks against Voice Recognition Systems: A Surveycited by 6
- 2024SpeechTaxi: On Multilingual Semantic Speech Classificationcited by 6
- 2020Adversarial attacks on audio source separationcited by 5
50 most cited of 64 All 64 (JSON)
Built on by
Citing papers per year: 2025: 1.
1 paper.
Similar, not cited
Close in wording, no citation either way
- 2026Adversarial Attacks on Multimodal Large Language Models: A Comprehensive Survey
- 2025Multifaceted Evaluation of Audio-Visual Capability for MLLMs: Effectiveness, Efficiency, Generalizability and Robustness
- 2025Survey of Adversarial Robustness in Multimodal Large Language Models
- 2025Beyond Text: Multimodal Jailbreaking of Vision-Language and Audio Models through Perceptually Simple Transformations
- 2024A Survey of Attacks on Large Vision-Language Models: Resources, Advances, and Future Trends
- 2024Unbridled Icarus: A Survey of the Potential Perils of Image Inputs in Multimodal Large Language Model Security
- 2024Seeing is Deceiving: Exploitation of Visual Pathways in Multi-Modal Language Models
- 2025Enhanced MLLM Black-Box Jailbreaking Attacks and Defenses