mlsanityData to 14 Oct 2026

Safe Inputs but Unsafe Output: Benchmarking Cross-modality Safety Alignment of Large Vision-Language Model

arXiv 2406.15279 ↗First posted 21 Jun 2024cs.AI cs.CLcited by 26

On the map: Multimodal LLMs / vision-language models → MLLM jailbreaks and adversarial robustness

also in 2 more
Abstract (arXiv)

As Artificial General Intelligence (AGI) becomes increasingly integrated into various facets of human life, ensuring the safety and ethical alignment of such systems is paramount. Previous studies primarily focus on single-modality threats, which may not suffice given the integrated and complex nature of cross-modality interactions. We introduce a novel safety alignment challenge called Safe Inputs but Unsafe Output (SIUO) to evaluate cross-modality safety alignment. Specifically, it considers cases where single modalities are safe independently but could potentially lead to unsafe or unethical outputs when combined. To empirically investigate this problem, we developed the SIUO, a cross-modality benchmark encompassing 9 critical safety domains, such as self-harm, illegal activities, and privacy violations. Our findings reveal substantial safety vulnerabilities in both closed- and open-source LVLMs, such as GPT-4V and LLaVA, underscoring the inadequacy of current models to reliably interpret and respond to complex, real-world scenarios.

Key facts. Cited by 26 papers in the map (data through 14 Oct 2026). Placed in MLLM jailbreaks and adversarial robustness, one of 16 directions in Multimodal LLMs / vision-language models. Builds on 21 of its 35 references that are in the map. First posted on arXiv 21 Jun 2024.

Builds on

21 of 35 references.

Closest ancestors · 6

Same direction, newest first

  1. 2024Cross-Modal Safety Alignment: Is textual unlearning all you need?27 May
  2. 2024Learning To See But Forgetting To Follow: Visual Instruction Tuning Makes LLMs More Prone To Jailbreak Attacks7 May
  3. 2024JailBreakV: A Benchmark for Assessing the Robustness of MultiModal Large Language Models against Jailbreak Attacks3 Apr
  4. 2024Safety of Multimodal Large Language Models on Images and Texts1 Feb
  5. 2024MLLM-Protector: Ensuring MLLM's Safety without Hurting Performance5 Jan
  6. 2023FigStep: Jailbreaking Large Vision-Language Models via Typographic Visual Prompts9 Nov

Other references · 15

  1. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,744
  2. 2023LLaMA: Open and Efficient Foundation Language Modelscited by 14,132
  3. 2023InstructBLIP: Towards General-purpose Vision-Language Models with Instruction Tuningcited by 2,876
  4. 2023MMMU: A Massive Multi-discipline Multimodal Understanding and Reasoning Benchmark for Expert AGIcited by 2,080
  5. 2023mPLUG-Owl: Modularization Empowers Large Language Models with Multimodalitycited by 1,040
  6. 2024Yi: Open Foundation Models by 01.AIcited by 719
  7. 2023mPLUG-Owl2: Revolutionizing Multi-modal Large Language Model with Modality Collaborationcited by 547
  8. 2023How Many Unicorns Are in This Image? A Safety Evaluation Benchmark for Vision LLMscited by 108
  9. 2023DRESS: Instructing Large Vision-Language Models to Align and Interact with Humans via Natural Language Feedbackcited by 103
  10. 2024InferAligner: Inference-Time Alignment for Harmlessness through Cross-Model Guidancecited by 83
  11. 2024GOAT-Bench: Safety Insights to Large Multimodal Models through Meme-Based Social Abusecited by 52
  12. 2024Assessment of Multimodal Large Language Models in Alignment with Human Valuescited by 36
3 more
  1. 2024Red Teaming GPT-4V: Are GPT-4V Safe Against Uni/Multi-Modal Jailbreak Attacks?cited by 34
  2. 2023Large Language Models as Automated Aligners for benchmarking Vision-Language Modelscited by 10
  3. 2020Egoshots, an ego-vision life-logging dataset and semantic fidelity metric to evaluate diversity in image captioning modelscited by 8

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2024: 1; 2025: 13; 2026: 12.

26 papers.

Sort:

By area:

  1. 2025A Comprehensive Survey in LLM(-Agent) Full Stack Safety: Data, Training and Deploymentcited by 116
  2. 2025A Survey of Safety on Large Vision-Language Models: Attacks, Defenses and Evaluationscited by 67
  3. 2025SafeGRPO: Self-Rewarded Multimodal Safety Alignment via Rule-Governed Policy Optimizationcited by 12
  4. 2025VLMGuard-R1: Proactive Safety Alignment for VLMs via Reasoning-Driven Prompt Optimizationcited by 10
  5. 2025SaFeR-VLM: Toward Safety-aware Fine-grained Reasoning in Multimodal Modelscited by 10
  6. 2024How Does Vision-Language Adaptation Impact the Safety of Vision Language Models?cited by 8
  7. 2025VLSU: Mapping the Limits of Joint Multimodal Understanding for AI Safetycited by 8
  8. 2025ShieldVLM: Safeguarding the Multimodal Implicit Toxicity via Deliberative Reasoning with LVLMscited by 7
  9. 2025Seeing the Threat: Vulnerabilities in Vision-Language Models to Adversarial Attackcited by 5
  10. 2026Evolving Contextual Safety in Multi-Modal Large Language Models via Inference-Time Self-Reflective Memorycited by 5
  11. 2025SIA: Enhancing Safety via Intent Awareness for Vision-Language Modelscited by 3
  12. 2026SafeRedir: Prompt Embedding Redirection for Robust Unlearning in Image Generation Modelscited by 3
14 more
  1. 2026Pragma-VL: Towards a Pragmatic Arbitration of Safety and Helpfulness in MLLMscited by 3
  2. 2025Differentially Private Federated Low Rank Adaptation Beyond Fixed-Matrixcited by 2
  3. 2025Agentic Moderation: Multi-Agent Design for Safer Vision-Language Modelscited by 2
  4. 2026Every Picture Tells a Dangerous Story: Memory-Augmented Multi-Agent Jailbreak Attacks on VLMscited by 2
  5. 2025Think-Reflect-Revise: A Policy-Guided Reflective Framework for Safety Alignment in Large Vision Language Modelscited by 1
  6. 2025COSMO-RL: Towards Trustworthy LMRMs via Joint Safety and Stabilitycited by 0
  7. 2026From Sparse Decisions to Dense Reasoning: A Multi-attribute Trajectory Paradigm for Multimodal Moderationcited by 0
  8. 2026OOD-MMSafe: Advancing MLLM Safety from Harmful Intent to Hidden Consequencescited by 0
  9. 2026V-DEAL: Diagnosing Video Safety De-Calibration as an Understanding-Refusal Coupling Failurecited by 0
  10. 2026Transfer Safety Awareness for Cross-Modal Safety Drift in Multimodal Large Language Modelscited by 0
  11. 2026A2Safe: Counterfactual Evidence-Aligned Adaptive Agent Collaboration for Safe and Effective Visual Question Answeringcited by 0
  12. 2026Can Vision-Language Models Stay Helpful When Facing Implicit Risks? Intent-Privilege OPSD for Efficient Safety-Helpfulness Alignmentcited by 0
  13. 2026MLLMs Fail to Refuse when Using Tools Agenticallycited by 0
  14. 2026From Local Evidence to Safety Verdicts: Causal Tracing in Vision-Language Modelscited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2025Understanding and Rectifying Safety Perception Distortion in VLMs
  2. 2024ETA: Evaluating Then Aligning Safety of Vision Language Models at Inference Time
  3. 2025Do We Really Need Curated Malicious Data for Safety Alignment in Multi-modal Large Language Models?
  4. 2026Evolving Safety Landscape of Multi-modal Large Language Models: A Survey of Emerging Threats and Safeguards
  5. 2025VisuoAlign: Safety Alignment of LVLMs with Multimodal Tree Search
  6. 2025Omni-SafetyBench: A Benchmark for Safety Evaluation of Audio-Visual Large Language Models
  7. 2024Unraveling and Mitigating Safety Alignment Degradation of Vision-Language Models
  8. 2025Safe RLHF-V: Safe Reinforcement Learning from Multi-modal Human Feedback