# Omni-SafetyBench: A Benchmark for Safety Evaluation of Audio-Visual Large Language Models arXiv: https://arxiv.org/abs/2508.07173 ยท HTML: https://mlsanity.com/paper/2508.07173/ ## Key facts - Cited by 18 papers in the map (data through 14 Oct 2026). - Placed in Vision-language and text-to-image safety, one of 12 directions in LLM safety and security. - Builds on 50 of its 57 references that are in the map. - First posted on arXiv 10 Aug 2025. ## Placed in - LLM safety and security / [Vision-language and text-to-image safety](https://mlsanity.com/safety/vision-language-and-text-to-image-safety/) ## Builds on (most cited first, 30 of 50) - [GPT-4 Technical Report](https://mlsanity.com/paper/2303.08774/) (2023) - [LLaMA: Open and Efficient Foundation Language Models](https://mlsanity.com/paper/2302.13971/) (2023) - [Direct Preference Optimization: Your Language Model is Secretly a Reward Model](https://mlsanity.com/paper/2305.18290/) (2023) - [Qwen2.5-VL Technical Report](https://mlsanity.com/paper/2502.13923/) (2025) - [GPT-4o System Card](https://mlsanity.com/paper/2410.21276/) (2024) - [SDXL: Improving Latent Diffusion Models for High-Resolution Image Synthesis](https://mlsanity.com/paper/2307.01952/) (2023) - [Gemini 2.5: Pushing the Frontier with Advanced Reasoning, Multimodality, Long Context, and Next Generation Agentic Capabilities](https://mlsanity.com/paper/2507.06261/) (2025) - [Training a Helpful and Harmless Assistant with Reinforcement Learning from Human Feedback](https://mlsanity.com/paper/2204.05862/) (2022) - [Gemini 1.5: Unlocking multimodal understanding across millions of tokens of context](https://mlsanity.com/paper/2403.05530/) (2024) - [Jailbroken: How Does LLM Safety Training Fail?](https://mlsanity.com/paper/2307.02483/) (2023) - [A Survey on LLM-as-a-Judge](https://mlsanity.com/paper/2411.15594/) (2024) - [MiniCPM-V: A GPT-4V Level MLLM on Your Phone](https://mlsanity.com/paper/2408.01800/) (2024) - [BeaverTails: Towards Improved Safety Alignment of LLM via a Human-Preference Dataset](https://mlsanity.com/paper/2307.04657/) (2023) - [VideoLLaMA 2: Advancing Spatial-Temporal Modeling and Audio Understanding in Video-LLMs](https://mlsanity.com/paper/2406.07476/) (2024) - [Qwen2.5-Omni Technical Report](https://mlsanity.com/paper/2503.20215/) (2025) - [Qwen2-Audio Technical Report](https://mlsanity.com/paper/2407.10759/) (2024) - [SALMONN: Towards Generic Hearing Abilities for Large Language Models](https://mlsanity.com/paper/2310.13289/) (2023) - [Safety Alignment Should Be Made More Than Just a Few Tokens Deep](https://mlsanity.com/paper/2406.05946/) (2024) - [Weak-to-Strong Generalization: Eliciting Strong Capabilities With Weak Supervision](https://mlsanity.com/paper/2312.09390/) (2023) - [FigStep: Jailbreaking Large Vision-Language Models via Typographic Visual Prompts](https://mlsanity.com/paper/2311.05608/) (2023) - [Great, Now Write an Article About That: The Crescendo Multi-Turn LLM Jailbreak Attack](https://mlsanity.com/paper/2404.01833/) (2024) - [Pyramidal Flow Matching for Efficient Video Generative Modeling](https://mlsanity.com/paper/2410.05954/) (2024) - [Unified-IO 2: Scaling Autoregressive Multimodal Models with Vision, Language, Audio, and Action](https://mlsanity.com/paper/2312.17172/) (2023) - [MM-SafetyBench: A Benchmark for Safety Evaluation of Multimodal Large Language Models](https://mlsanity.com/paper/2311.17600/) (2023) - [AudioGPT: Understanding and Generating Speech, Music, Sound, and Talking Head](https://mlsanity.com/paper/2304.12995/) (2023) - [PKU-SafeRLHF: Towards Multi-Level Safety Alignment for LLMs with Human Preference](https://mlsanity.com/paper/2406.15513/) (2024) - [Jailbreak Attacks and Defenses Against Large Language Models: A Survey](https://mlsanity.com/paper/2407.04295/) (2024) - [SALAD-Bench: A Hierarchical and Comprehensive Safety Benchmark for Large Language Models](https://mlsanity.com/paper/2402.05044/) (2024) - [Mini-Omni: Language Models Can Hear, Talk While Thinking in Streaming](https://mlsanity.com/paper/2408.16725/) (2024) - [SafetyBench: Evaluating the Safety of Large Language Models](https://mlsanity.com/paper/2309.07045/) (2023) ## Built on by (most cited first, 18 of 18) - [SocialOmni: Benchmarking Audio-Visual Social Interactivity in Omni Models](https://mlsanity.com/paper/2603.16859/) (2026) - [MicroRemed: Benchmarking LLMs in Microservices Remediation](https://mlsanity.com/paper/2511.01166/) (2025) - [VoxSafeBench: Not Just What Is Said, but Who, How, and Where](https://mlsanity.com/paper/2604.14548/) (2026) - [Omni-Safety under Cross-Modality Conflict: Vulnerabilities, Dynamics Mechanisms and Efficient Alignment](https://mlsanity.com/paper/2602.10161/) (2026) - [Efficient Failure Management for Multi-Agent Systems with Reasoning Trace Representation](https://mlsanity.com/paper/2603.21522/) (2026) - [From Feedback Loops to Policy Updates: Reinforcement Fine-Tuning for LLM-Based Alpha Factor Discovery](https://mlsanity.com/paper/2605.15412/) (2026) - [AudioGuard: Toward Comprehensive Audio Safety Protection Across Diverse Threat Models](https://mlsanity.com/paper/2604.08867/) (2026) - [Towards Robust LLM Post-Training: Automatic Failure Management for Reinforcement Fine-Tuning](https://mlsanity.com/paper/2605.04431/) (2026) - [Towards In-Depth Root Cause Localization for Microservices with Multi-Agent Recursion-of-Thought](https://mlsanity.com/paper/2605.14866/) (2026) - [Safety Geometry Collapse in Multimodal LLMs and Adaptive Drift Correction](https://mlsanity.com/paper/2605.18104/) (2026) - [Audio Jailbreaks in Large Audio-Language Models: Taxonomy, Attack-Defense Analysis, and Cost-Aware Evaluation](https://mlsanity.com/paper/2605.30031/) (2026) - [RedVox: Safety and Fairness Gaps in Speech Models Across Languages](https://mlsanity.com/paper/2606.26968/) (2026) - [V-DEAL: Diagnosing Video Safety De-Calibration as an Understanding-Refusal Coupling Failure](https://mlsanity.com/paper/2607.21151/) (2026) - [MMJailBench: A Factorized Benchmark for Disentangling Multimodal Jailbreak Vulnerabilities](https://mlsanity.com/paper/2608.25490/) (2026) - [Multi2AV-Safety: Benchmarking Safety in Multimodal-to-Audio-Video Generation](https://mlsanity.com/paper/2608.26535/) (2026) - [Playing social deduction games with reinforcement fine-tuned large language models](https://mlsanity.com/paper/2610.04261/) (2026) - [From Acoustic Sensing to Physical Actions: A Comprehensive Survey of Security Risks in Edge-Deployed Audio LLM-Based Agents](https://mlsanity.com/paper/2610.15390/) (2026) - [Safety for Whom? Benchmarking Understanding Beyond Refusal in Egocentric Multimodal Assistants](https://mlsanity.com/paper/2610.17728/) (2026) ## Abstract (arXiv) Omni-modal Large Language Models (OLLMs) that integrate visual, auditory, and textual processing face severe safety risks. They exhibit fragile defenses against audio-visual joint harmful inputs and demonstrate inconsistent safety performance across different modalities, enabling simple modality-switching jailbreaks. However, existing safety benchmarks fail to comprehensively assess these risks due to the absence of audio-visual joint samples, limited modality coverage, and lack of parallel test cases for cross-modal consistency evaluation. To address these gaps, we introduce Omni-SafetyBench, the first comprehensive parallel benchmark for OLLM safety evaluation, featuring 23,328 test instances across 24 modality variations derived from 972 seed samples. Recognizing that complex inputs pose comprehension challenges and that cross-modal consistency is critical for OLLM safety, we propose tailored metrics: a Safety-score based on Conditional Attack Success Rate (C-ASR) and Conditional Refusal Rate (C-RR), and a Cross-Modal Safety Consistency score (CMSC-score). Evaluating 11 state-of-the-art OLLMs reveals severe vulnerabilities: only 3 models exceed 0.6 in both metrics, with safety degrading sharply for audio-visual inputs. Furthermore, evaluation of existing safety alignment methods on Omni-SafetyBench identifies fundamental challenges in OLLM safety alignment, highlighting urgent needs for enhanced research in this domain.