mlsanityData to 14 Oct 2026

Omni-SafetyBench: A Benchmark for Safety Evaluation of Audio-Visual Large Language Models

arXiv 2508.07173 ↗First posted 10 Aug 2025cs.CLcited by 18

On the map: LLM safety and security → Vision-language and text-to-image safety

Abstract (arXiv)

Omni-modal Large Language Models (OLLMs) that integrate visual, auditory, and textual processing face severe safety risks. They exhibit fragile defenses against audio-visual joint harmful inputs and demonstrate inconsistent safety performance across different modalities, enabling simple modality-switching jailbreaks. However, existing safety benchmarks fail to comprehensively assess these risks due to the absence of audio-visual joint samples, limited modality coverage, and lack of parallel test cases for cross-modal consistency evaluation. To address these gaps, we introduce Omni-SafetyBench, the first comprehensive parallel benchmark for OLLM safety evaluation, featuring 23,328 test instances across 24 modality variations derived from 972 seed samples. Recognizing that complex inputs pose comprehension challenges and that cross-modal consistency is critical for OLLM safety, we propose tailored metrics: a Safety-score based on Conditional Attack Success Rate (C-ASR) and Conditional Refusal Rate (C-RR), and a Cross-Modal Safety Consistency score (CMSC-score). Evaluating 11 state-of-the-art OLLMs reveals severe vulnerabilities: only 3 models exceed 0.6 in both metrics, with safety degrading sharply for audio-visual inputs. Furthermore, evaluation of existing safety alignment methods on Omni-SafetyBench identifies fundamental challenges in OLLM safety alignment, highlighting urgent needs for enhanced research in this domain.

Key facts. Cited by 18 papers in the map (data through 14 Oct 2026). Placed in Vision-language and text-to-image safety, one of 12 directions in LLM safety and security. Builds on 50 of its 57 references that are in the map. First posted on arXiv 10 Aug 2025.

Builds on

50 of 57 references.

Closest ancestors · 6

Same direction, newest first

  1. 2025Understanding and Rectifying Safety Perception Distortion in VLMs18 Feb
  2. 2025SEA: Low-Resource Safety Alignment for Multimodal Large Language Models via Synthetic Embeddings18 Feb
  3. 2024SPA-VL: A Comprehensive Safety Preference Alignment Dataset for Vision Language Model17 Jun
  4. 2024AdaShield: Safeguarding Multimodal Large Language Models from Structure-based Attack via Adaptive Shield Prompting14 Mar
  5. 2024Safety Fine-Tuning at (Almost) No Cost: A Baseline for Vision Large Language Models3 Feb
  6. 2023FigStep: Jailbreaking Large Vision-Language Models via Typographic Visual Prompts9 Nov

Other references · 44

  1. 2023GPT-4 Technical Reportcited by 18,396
  2. 2023LLaMA: Open and Efficient Foundation Language Modelscited by 14,132
  3. 2023Direct Preference Optimization: Your Language Model is Secretly a Reward Modelcited by 8,065
  4. 2025Qwen2.5-VL Technical Reportcited by 5,587
  5. 2024GPT-4o System Cardcited by 4,588
  6. 2023SDXL: Improving Latent Diffusion Models for High-Resolution Image Synthesiscited by 4,356
  7. 2025Gemini 2.5: Pushing the Frontier with Advanced Reasoning, Multimodality, Long Context, and Next Generation Agentic Capabilitiescited by 3,796
  8. 2022Training a Helpful and Harmless Assistant with Reinforcement Learning from Human Feedbackcited by 3,720
  9. 2024Gemini 1.5: Unlocking multimodal understanding across millions of tokens of contextcited by 3,224
  10. 2023Jailbroken: How Does LLM Safety Training Fail?cited by 1,651
  11. 2024A Survey on LLM-as-a-Judgecited by 1,423
  12. 2024MiniCPM-V: A GPT-4V Level MLLM on Your Phonecited by 1,079
32 more
  1. 2023BeaverTails: Towards Improved Safety Alignment of LLM via a Human-Preference Datasetcited by 860
  2. 2024VideoLLaMA 2: Advancing Spatial-Temporal Modeling and Audio Understanding in Video-LLMscited by 771
  3. 2025Qwen2.5-Omni Technical Reportcited by 736
  4. 2024Qwen2-Audio Technical Reportcited by 560
  5. 2023SALMONN: Towards Generic Hearing Abilities for Large Language Modelscited by 506
  6. 2024Safety Alignment Should Be Made More Than Just a Few Tokens Deepcited by 472
  7. 2023Weak-to-Strong Generalization: Eliciting Strong Capabilities With Weak Supervisioncited by 467
  8. 2024Great, Now Write an Article About That: The Crescendo Multi-Turn LLM Jailbreak Attackcited by 390
  9. 2024Pyramidal Flow Matching for Efficient Video Generative Modelingcited by 366
  10. 2023Unified-IO 2: Scaling Autoregressive Multimodal Models with Vision, Language, Audio, and Actioncited by 302
  11. 2023MM-SafetyBench: A Benchmark for Safety Evaluation of Multimodal Large Language Modelscited by 274
  12. 2023AudioGPT: Understanding and Generating Speech, Music, Sound, and Talking Headcited by 265
  13. 2024PKU-SafeRLHF: Towards Multi-Level Safety Alignment for LLMs with Human Preferencecited by 261
  14. 2024Jailbreak Attacks and Defenses Against Large Language Models: A Surveycited by 230
  15. 2024SALAD-Bench: A Hierarchical and Comprehensive Safety Benchmark for Large Language Modelscited by 228
  16. 2024Mini-Omni: Language Models Can Hear, Talk While Thinking in Streamingcited by 210
  17. 2023SafetyBench: Evaluating the Safety of Large Language Modelscited by 199
  18. 2025VITA-1.5: Towards GPT-4o Level Real-Time Vision and Speech Interactioncited by 197
  19. 2024Eyes Closed, Safety On: Protecting Multimodal LLMs via Image-to-Text Transformationcited by 124
  20. 2025Baichuan-Omni-1.5 Technical Reportcited by 111
  21. 2024Mini-Omni2: Towards Open-source GPT-4o with Vision, Speech and Duplex Capabilitiescited by 109
  22. 2024OmniBench: Towards The Future of Universal Omni-Language Modelscited by 102
  23. 2024DrAttack: Prompt Decomposition and Reconstruction Makes Powerful LLM Jailbreakerscited by 101
  24. 2024ETA: Evaluating Then Aligning Safety of Vision Language Models at Inference Timecited by 60
  25. 2024Immune: Improving Safety Against Jailbreaks in Multi-modal LLMs via Inference-Time Alignmentcited by 44
  26. 2024TIS-DPO: Token-level Importance Sampling for Direct Preference Optimization With Estimated Weightscited by 42
  27. 2024Audio Is the Achilles' Heel: Red Teaming Audio Large Multimodal Modelscited by 37
  28. 2024CoCA: Regaining Safety-awareness of Multimodal Large Language Models with Constitutional Calibrationcited by 31
  29. 2024WordGame: Efficient & Effective LLM Jailbreak via Simultaneous Obfuscation in Query and Responsecited by 29
  30. 2025Video-SafetyBench: A Benchmark for Safety Evaluation of Video LVLMscited by 28
  31. 2024Direct Large Language Model Alignment Through Self-Rewarding Contrastive Prompt Distillationcited by 18
  32. 2025AudioTrust: Benchmarking the Multifaceted Trustworthiness of Audio Large Language Modelscited by 14

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2025: 1; 2026: 17.

18 papers.

Sort:

By area:

  1. 2026SocialOmni: Benchmarking Audio-Visual Social Interactivity in Omni Modelscited by 13
  2. 2025MicroRemed: Benchmarking LLMs in Microservices Remediationcited by 11
  3. 2026VoxSafeBench: Not Just What Is Said, but Who, How, and Wherecited by 8
  4. 2026Omni-Safety under Cross-Modality Conflict: Vulnerabilities, Dynamics Mechanisms and Efficient Alignmentcited by 6
  5. 2026Efficient Failure Management for Multi-Agent Systems with Reasoning Trace Representationcited by 6
  6. 2026From Feedback Loops to Policy Updates: Reinforcement Fine-Tuning for LLM-Based Alpha Factor Discoverycited by 6
  7. 2026AudioGuard: Toward Comprehensive Audio Safety Protection Across Diverse Threat Modelscited by 3
  8. 2026Towards Robust LLM Post-Training: Automatic Failure Management for Reinforcement Fine-Tuningcited by 3
  9. 2026Towards In-Depth Root Cause Localization for Microservices with Multi-Agent Recursion-of-Thoughtcited by 1
  10. 2026Safety Geometry Collapse in Multimodal LLMs and Adaptive Drift Correctioncited by 1
  11. 2026Audio Jailbreaks in Large Audio-Language Models: Taxonomy, Attack-Defense Analysis, and Cost-Aware Evaluationcited by 1
  12. 2026RedVox: Safety and Fairness Gaps in Speech Models Across Languagescited by 0
6 more
  1. 2026V-DEAL: Diagnosing Video Safety De-Calibration as an Understanding-Refusal Coupling Failurecited by 0
  2. 2026MMJailBench: A Factorized Benchmark for Disentangling Multimodal Jailbreak Vulnerabilitiescited by 0
  3. 2026Multi2AV-Safety: Benchmarking Safety in Multimodal-to-Audio-Video Generationcited by 0
  4. 2026Playing social deduction games with reinforcement fine-tuned large language modelscited by 0
  5. 2026From Acoustic Sensing to Physical Actions: A Comprehensive Survey of Security Risks in Edge-Deployed Audio LLM-Based Agentscited by 0
  6. 2026Safety for Whom? Benchmarking Understanding Beyond Refusal in Egocentric Multimodal Assistantscited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2026MCBench: A Multicontext Safety Assessment Benchmark for Omni Large Language Models
  2. 2026CSR-Bench: A Benchmark for Evaluating the Cross-modal Safety and Reliability of MLLMs
  3. 2026Transfer Safety Awareness for Cross-Modal Safety Drift in Multimodal Large Language Models
  4. 2025Can't See the Forest for the Trees: Benchmarking Multimodal Safety Awareness for Multimodal LLMs
  5. 2026Boosting Omni-Modal Language Models: Staged Post-Training with Visually Debiased Evaluation
  6. 2026Evolving Safety Landscape of Multi-modal Large Language Models: A Survey of Emerging Threats and Safeguards
  7. 2026Benign Fine-Tuning Breaks Safety Alignment in Audio LLMs
  8. 2025Do We Really Need Curated Malicious Data for Safety Alignment in Multi-modal Large Language Models?