mlsanityData to 14 Oct 2026

V-DEAL: Diagnosing Video Safety De-Calibration as an Understanding-Refusal Coupling Failure

arXiv 2607.21151 ↗First posted 23 Jul 2026cs.AIcited by 0

On the map: LLM alignment / RLHF / preference optimisation → Safety alignment and jailbreak attacks

also in 1 more
Abstract (arXiv)

As Video Large Language Models are increasingly deployed in real-world applications, ensuring their safety alignment has become critical. Counterintuitively, we find that harmful videos paired with benign queries achieve higher attack success rates than the same videos paired with explicitly harmful queries. To understand the underlying mechanism of this vulnerability, we present V-DEAL, a three-level diagnostic framework that jointly analyzes this failure across model behaviour, understanding, and internal representations. By progressively ruling out perception failure and quantifying the model's internal refusal tendency, V-DEAL provides a new diagnostic perspective for analyzing the underlying mechanism of the observed vulnerability. We tested six Video LLMs on three public benchmarks and observed that models correctly recognize harmful video content with over 81% accuracy, yet the average attack success rate still reaches 48.33% under the condition pairing harmful videos with benign queries. Hidden-state analysis further shows that visual understanding activates a weaker refusal tendency than textual understanding. Furthermore, we introduce a prompt injection intervention method that reduces attack success rates by an average of 48.24 percentage points and achieves performance comparable to prior fine-tuning-based methods, providing an effective and practical means to address such safety risks in Video LLMs.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Placed in Safety alignment and jailbreak attacks, one of 10 directions in LLM alignment / RLHF / preference optimisation. Builds on 61 of its 66 references that are in the map. First posted on arXiv 23 Jul 2026.

Builds on

61 of 66 references.

Closest ancestors · 7

Same direction, newest first

  1. 2025Safety Subspaces are Not Linearly Distinct: A Fine-Tuning Case Study20 May
  2. 2025SEA: Low-Resource Safety Alignment for Multimodal Large Language Models via Synthetic Embeddings18 Feb
  3. 2025VLM-Guard: Safeguarding Vision-Language Models via Fulfilling Safety Alignment Gap14 Feb
  4. 2024ETA: Evaluating Then Aligning Safety of Vision Language Models at Inference Time9 Oct
  5. 2024Safe Inputs but Unsafe Output: Benchmarking Cross-modality Safety Alignment of Large Vision-Language Model21 Jun
  6. 2024SPA-VL: A Comprehensive Safety Preference Alignment Dataset for Vision Language Model17 Jun
  7. 2023FigStep: Jailbreaking Large Vision-Language Models via Typographic Visual Prompts9 Nov

Other references · 54

  1. 2024Qwen2-VL: Enhancing Vision-Language Model's Perception of the World at Any Resolutioncited by 4,234
  2. 2025Qwen3-VL Technical Reportcited by 2,793
  3. 2025InternVL3.5: Advancing Open-Source Multimodal Models in Versatility, Reasoning, and Efficiencycited by 1,510
  4. 2023Representation Engineering: A Top-Down Approach to AI Transparencycited by 1,336
  5. 2024Refusal in Language Models Is Mediated by a Single Directioncited by 1,032
  6. 2023Vision-Language Models for Vision Tasks: A Surveycited by 839
  7. 2023XSTest: A Test Suite for Identifying Exaggerated Safety Behaviours in Large Language Modelscited by 505
  8. 2023MM-SafetyBench: A Benchmark for Safety Evaluation of Multimodal Large Language Modelscited by 274
  9. 2024SafeDecoding: Defending against Jailbreak Attacks via Safety-Aware Decodingcited by 247
  10. 2024JailBreakV: A Benchmark for Assessing the Robustness of MultiModal Large Language Models against Jailbreak Attackscited by 230
  11. 2025InternVideo2.5: Empowering Video MLLMs with Long and Rich Context Modelingcited by 211
  12. 2025MiniCPM-V 4.5: Cooking Efficient MLLMs via Architecture, Data, and Training Recipecited by 186
38 more
  1. 2024Safety Fine-Tuning at (Almost) No Cost: A Baseline for Vision Large Language Modelscited by 167
  2. 2024AdaShield: Safeguarding Multimodal Large Language Models from Structure-based Attack via Adaptive Shield Promptingcited by 125
  3. 2026MiniCPM-o 4.5: Towards Real-Time Full-Duplex Omni-Modal Interactioncited by 78
  4. 2024Safety of Multimodal Large Language Models on Images and Textscited by 75
  5. 2024Visual-RolePlay: Universal Jailbreak Attack on MultiModal Large Language Models via Role-playing Image Charactercited by 68
  6. 2024VLSBench: Unveiling Visual Leakage in Multimodal Safetycited by 63
  7. 2025GuardReasoner-VL: Safeguarding VLMs via Reinforced Reasoningcited by 42
  8. 2024IDEATOR: Jailbreaking and Benchmarking Large Vision-Language Models Using Themselvescited by 37
  9. 2025HiddenDetect: Detecting Jailbreak Attacks against Large Vision-Language Models via Monitoring Hidden Statescited by 33
  10. 2024Jailbreak Attacks and Defenses against Multimodal Generative Models: A Surveycited by 32
  11. 2025DiMo-GUI: Advancing Test-time Scaling in GUI Grounding via Modality-Aware Visual Reasoningcited by 31
  12. 2023BrainVis: Exploring the Bridge between Brain and Visual Signals via Image Reconstructioncited by 30
  13. 2024BlueSuffix: Reinforced Blue Teaming for Vision-Language Models Against Jailbreak Attackscited by 28
  14. 2025Video-SafetyBench: A Benchmark for Safety Evaluation of Video LVLMscited by 28
  15. 2024"Not Aligned" is Not "Malicious": Being Careful about Hallucinations of Large Language Models' Jailbreakcited by 25
  16. 2024SLANG: New Concept Comprehension of Large Language Modelscited by 20
  17. 2025Innate Reasoning is Not Enough: In-Context Learning Enhances Reasoning Large Language Models with Less Overthinkingcited by 20
  18. 2025Omni-SafetyBench: A Benchmark for Safety Evaluation of Audio-Visual Large Language Modelscited by 18
  19. 2024HiddenGuard: Fine-Grained Safe Generation with Specialized Representation Routercited by 16
  20. 2024You Know What I'm Saying: Jailbreak Attack via Implicit Referencecited by 14
  21. 2025VistaWise: Building Cost-Effective Agent with Cross-Modal Knowledge Graph for Minecraftcited by 13
  22. 2024Defending Multimodal Backdoored Models by Repulsive Visual Prompt Tuningcited by 12
  23. 2025MSTS: A Multimodal Safety Test Suite for Vision-Language Modelscited by 12
  24. 2025RefineShot: Rethinking Cinematography Understanding with Foundational Skill Evaluationcited by 10
  25. 2025ContextNav: Towards Agentic Multimodal In-Context Learningcited by 10
  26. 2025MRFD: Multi-Region Fusion Decoding with Self-Consistency for Mitigating Hallucinations in LVLMscited by 9
  27. 2026CamReasoner: Reinforcing Camera Movement Understanding via Structured Spatial Reasoningcited by 9
  28. 2024Effective and Efficient Adversarial Detection for Vision-Language Models via A Single Vectorcited by 8
  29. 2025Focusing by Contrastive Attention: Enhancing VLMs' Visual Reasoningcited by 8
  30. 2025VLSU: Mapping the Limits of Joint Multimodal Understanding for AI Safetycited by 8
  31. 2025Spot Risks Before Speaking! Unraveling Safety Attention Heads in Large Vision-Language Modelscited by 7
  32. 2025MSR-Align: Policy-Grounded Multimodal Alignment for Safety-Aware Reasoning in Vision-Language Modelscited by 7
  33. 2025WaMo: Wavelet-Enhanced Multi-Frequency Trajectory Analysis for Fine-Grained Text-Motion Retrievalcited by 7
  34. 2026What Should a Streaming Video Model Remember?cited by 7
  35. 2024Tuning Vision-Language Models with Candidate Labels by Prompt Alignmentcited by 6
  36. 2026Gated Differentiable Working Memory for Long-Context Language Modelingcited by 6
  37. 2026VideoStir: Understanding Long Videos via Spatio-Temporally Structured and Intent-Aware RAGcited by 6
  38. 2025Enhanced Cross-modal 3D Retrieval via Tri-modal Reconstructioncited by 5

50 most cited of 54 All 54 (JSON)

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2025From Evaluation to Defense: Advancing Safety in Video Large Language Models
  2. 2026VirtueBench: Evaluating Trustworthiness under Uncertainty in Long Video Understanding
  3. 2026Jailbreaking Multimodal Large Language Models using Multi-Clip Video
  4. 2026HarmVideoBench: Benchmarking Harmful Video Understanding in Large Multimodal Models
  5. 2025DUAL-Bench: Measuring Over-Refusal and Robustness in Vision-Language Models
  6. 2025Failures to Surface Harmful Contents in Video Large Language Models
  7. 2025Security Tensors as a Cross-Modal Bridge: Extending Text-Aligned Safety to Vision in LVLM
  8. 2026Scenes as an Attack Surface: Uncovering Scene-Grounded Safety Failures in MLLMs