mlsanityData to 7 Oct 2026

Diagnosing and Repairing Unsafe Channels in Vision-Language Models via Causal Discovery and Dual-Modal Safety Subspace Projection

arXiv 2603.27240 ↗First posted 28 Mar 2026cs.CV cs.AIcited by 1

On the map: Multimodal LLMs / vision-language models → MLLM jailbreaks and adversarial robustness

Abstract (arXiv)

Large Vision-Language Models (LVLMs) have achieved impressive performance across multimodal understanding and reasoning tasks, yet their internal safety mechanisms remain opaque and poorly controlled. In this work, we present a comprehensive framework for diagnosing and repairing unsafe channels within LVLMs (CARE). We first perform causal mediation analysis to identify neurons and layers that are causally responsible for unsafe behaviors. Based on these findings, we introduce a dual-modal safety subspace projection method that learns generalized safety subspaces for both visual and textual modalities through generalized eigen-decomposition between benign and malicious activations. During inference, activations are dynamically projected toward these safety subspaces via a hybrid fusion mechanism that adaptively balances visual and textual corrections, effectively suppressing unsafe features while preserving semantic fidelity. Extensive experiments on multiple safety benchmarks demonstrate that our causal-subspace repair framework significantly enhances safety robustness without degrading general multimodal capabilities, outperforming prior activation steering and alignment-based baselines. Additionally, our method exhibits good transferability, defending against unseen attacks.

Key facts. Cited by 1 paper in the map (data through 7 Oct 2026). Placed in MLLM jailbreaks and adversarial robustness, one of 16 directions in Multimodal LLMs / vision-language models. Builds on 29 of its 57 references that are in the map. First posted on arXiv 28 Mar 2026.

Builds on

29 of 57 references.

Closest ancestors · 9

Same direction, newest first

  1. 2025Activation Steering Meets Preference Optimization: Defense Against Jailbreaks in Vision Language Models30 Aug
  2. 2025Understanding and Rectifying Safety Perception Distortion in VLMs18 Feb
  3. 2025Universal Adversarial Attack on Aligned Multimodal LLMs11 Feb
  4. 2024JailbreakZoo: Survey, Landscapes, and Horizons in Jailbreaking Large Language and Vision-Language Models26 Jun
  5. 2024JailBreakV: A Benchmark for Assessing the Robustness of MultiModal Large Language Models against Jailbreak Attacks3 Apr
  6. 2024Red Teaming Visual Language Models23 Jan
  7. 2023MM-SafetyBench: A Benchmark for Safety Evaluation of Multimodal Large Language Models29 Nov
  8. 2023Jailbreak in pieces: Compositional Adversarial Attacks on Multi-Modal Language Models26 Jul
  9. 2023Abusing Images and Sounds for Indirect Instruction Injection in Multi-Modal LLMs19 Jul

Other references · 20

  1. 2017Towards Deep Learning Models Resistant to Adversarial Attackscited by 6,326
  2. 2025Qwen2.5-VL Technical Reportcited by 5,418
  3. 2023Universal and Transferable Adversarial Attacks on Aligned Language Modelscited by 2,914
  4. 2024LLaVA-OneVision: Easy Visual Task Transfercited by 2,669
  5. 2024HarmBench: A Standardized Evaluation Framework for Automated Red Teaming and Robust Refusalcited by 1,330
  6. 2023MM-Vet: Evaluating Large Multimodal Models for Integrated Capabilitiescited by 1,221
  7. 2016Adversarial Machine Learning at Scalecited by 934
  8. 2023Steering Language Models With Activation Engineeringcited by 824
  9. 2024DeepSeek-VL: Towards Real-World Vision-Language Understandingcited by 718
  10. 2022Red Teaming Language Models to Reduce Harms: Methods, Scaling Behaviors, and Lessons Learnedcited by 702
  11. 2022Diffusion Models for Adversarial Purificationcited by 469
  12. 2024OR-Bench: An Over-Refusal Benchmark for Large Language Modelscited by 222
8 more
  1. 2024Programming Refusal with Conditional Activation Steeringcited by 191
  2. 2024A Mechanistic Understanding of Alignment Algorithms: A Case Study on DPO and Toxicitycited by 179
  3. 2024Jailbreaking Attack against Multimodal Large Language Modelcited by 128
  4. 2023JailGuard: A Universal Detection Framework for LLM Prompt-based Attackscited by 46
  5. 2019Eigenvalue and Generalized Eigenvalue Problems: Tutorialcited by 45
  6. 2025On the Trustworthiness of Generative Foundation Models: Guideline, Assessment, and Perspectivecited by 39
  7. 2024Model Editing as a Robust and Denoised variant of DPO: A Case Study on Toxicitycited by 33
  8. 2025Why Pre-trained Models Fail: Feature Entanglement in Multi-modal Depression Detectioncited by 2

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2026: 1.

1 paper.

Similar, not cited

Close in wording, no citation either way

  1. 2026Do VLMs Share Safety Neurons Across Modalities?
  2. 2025Security Tensors as a Cross-Modal Bridge: Extending Text-Aligned Safety to Vision in LVLM
  3. 2026One Anchor for All: Unified Multilingual and Multimodal Safety Alignment for LVLMs
  4. 2025Self-Aware Safety Augmentation: Leveraging Internal Semantic Understanding to Enhance Safety in Vision-Language Models
  5. 2026Targeted Interpretable Safety Neuron Enhancement for Multilingual Vision-Language Large Models
  6. 2026Risk Awareness Injection: Calibrating Vision-Language Models for Safety without Compromising Utility
  7. 2026Two Birds, One Projection: Harmonizing Safety and Utility in LVLMs via Inference-time Feature Projection
  8. 2025Safety Mirage: How Spurious Correlations Undermine VLM Safety Fine-Tuning and Can Be Mitigated by Machine Unlearning