mlsanityData to 14 Oct 2026

Contextual Image Attack: How Visual Context Exposes Multimodal Safety Vulnerabilities

arXiv 2512.02973 ↗First posted 2 Dec 2025cs.CV cs.CL cs.CRcited by 3

On the map: Multimodal LLMs / vision-language models → MLLM jailbreaks and adversarial robustness

also in 1 more
Abstract (arXiv)

While Multimodal Large Language Models (MLLMs) show remarkable capabilities, their safety alignments are susceptible to jailbreak attacks. Existing attack methods typically focus on text-image interplay, treating the visual modality as a secondary prompt. This approach underutilizes the unique potential of images to carry complex, contextual information. To address this gap, we propose a new image-centric attack method, Contextual Image Attack (CIA), which employs a multi-agent system to subtly embeds harmful queries into seemingly benign visual contexts using four distinct visualization strategies. To further enhance the attack's efficacy, the system incorporate contextual element enhancement and automatic toxicity obfuscation techniques. Experimental results on the MMSafetyBench-tiny dataset show that CIA achieves high toxicity scores of 4.73 and 4.83 against the GPT-4o and Qwen2.5-VL-72B models, respectively, with Attack Success Rates (ASR) reaching 86.31% and 91.07%. Our method significantly outperforms prior work, demonstrating that the visual modality itself is a potent vector for jailbreaking advanced MLLMs.

Key facts. Cited by 3 papers in the map (data through 14 Oct 2026). Placed in MLLM jailbreaks and adversarial robustness, one of 16 directions in Multimodal LLMs / vision-language models. Builds on 47 of its 57 references that are in the map. First posted on arXiv 2 Dec 2025.

Builds on

47 of 57 references.

Closest ancestors · 12

Same direction, newest first

  1. 2025Self-adaptive Dataset Construction for Real-World Multimodal Safety Scenarios4 Sep
  2. 2025PiCo: Jailbreaking Multimodal Large Language Models via Pictorial Code Contextualization2 Apr
  3. 2025Survey of Adversarial Robustness in Multimodal Large Language Models18 Mar
  4. 2025Rethinking Bottlenecks in Safety Fine-Tuning of Vision Language Models30 Jan
  5. 2025Jailbreaking Multimodal Large Language Models via Shuffle Inconsistency9 Jan
  6. 2024Cross-modality Information Check for Detecting Jailbreaking in Multimodal Large Language Models31 Jul
  7. 2024MultiTrust: A Comprehensive Benchmark Towards Trustworthy Multimodal Large Language Models11 Jun
  8. 2024Cross-Modal Safety Alignment: Is textual unlearning all you need?27 May
  9. 2023MM-SafetyBench: A Benchmark for Safety Evaluation of Multimodal Large Language Models29 Nov
  10. 2023FigStep: Jailbreaking Large Vision-Language Models via Typographic Visual Prompts9 Nov
  11. 2023Visual Adversarial Examples Jailbreak Aligned Large Language Models22 Jun
  12. 2023On Evaluating Adversarial Robustness of Large Vision-Language Models26 May

Other references · 35

  1. 2021High-Resolution Image Synthesis with Latent Diffusion Modelscited by 18,863
  2. 2025Qwen2.5-VL Technical Reportcited by 5,587
  3. 2023Improved Baselines with Visual Instruction Tuningcited by 4,418
  4. 2025Gemini 2.5: Pushing the Frontier with Advanced Reasoning, Multimodality, Long Context, and Next Generation Agentic Capabilitiescited by 3,796
  5. 2023MiniGPT-4: Enhancing Vision-Language Understanding with Advanced Large Language Modelscited by 2,711
  6. 2023A Survey of Large Language Modelscited by 2,576
  7. 2015VQA: Visual Question Answeringcited by 2,506
  8. 2023Video-LLaMA: An Instruction-tuned Audio-Visual Language Model for Video Understandingcited by 1,471
  9. 2023A Survey on Evaluation of Large Language Modelscited by 1,242
  10. 2023ImageBind: One Embedding Space To Bind Them Allcited by 1,175
  11. 2023Fine-tuning Aligned Language Models Compromises Safety, Even When Users Do Not Intend To!cited by 1,154
  12. 2023A Survey on Multimodal Large Language Modelscited by 909
23 more
  1. 2023Vision-Language Models for Vision Tasks: A Surveycited by 839
  2. 2023AutoDAN: Generating Stealthy Jailbreak Prompts on Aligned Large Language Modelscited by 785
  3. 2018From Recognition to Cognition: Visual Commonsense Reasoningcited by 556
  4. 2023"Do Anything Now": Characterizing and Evaluating In-The-Wild Jailbreak Prompts on Large Language Modelscited by 489
  5. 2024Jailbreak Attacks and Defenses Against Large Language Models: A Surveycited by 230
  6. 2024SALAD-Bench: A Hierarchical and Comprehensive Safety Benchmark for Large Language Modelscited by 228
  7. 2024An Introduction to Vision-Language Modelingcited by 178
  8. 2023Image Hijacks: Adversarial Images can Control Generative Models at Runtimecited by 172
  9. 2024Safety Fine-Tuning at (Almost) No Cost: A Baseline for Vision Large Language Modelscited by 167
  10. 2021VisualMRC: Machine Reading Comprehension on Document Imagescited by 151
  11. 2024Jailbreaking Attack against Multimodal Large Language Modelcited by 135
  12. 2024Visual-RolePlay: Universal Jailbreak Attack on MultiModal Large Language Models via Role-playing Image Charactercited by 68
  13. 2023JailGuard: A Universal Detection Framework for LLM Prompt-based Attackscited by 50
  14. 2025T2ISafety: Benchmark for Assessing Fairness, Toxicity, and Privacy in Image Generationcited by 48
  15. 2023EVCap: Retrieval-Augmented Image Captioning with External Visual-Name Memory for Open-World Comprehensioncited by 32
  16. 2024The First to Know: How Token Distributions Reveal Hidden Knowledge in Large Vision-Language Models?cited by 26
  17. 2025Layer-Aware Representation Filtering: Purifying Finetuning Data to Preserve LLM Safety Alignmentcited by 24
  18. 2023Prophet: Prompting Large Language Models with Complementary Answer Heuristics for Knowledge-based Visual Question Answeringcited by 21
  19. 2023Q: How to Specialize Large Vision-Language Models to Data-Scarce VQA Tasks? A: Self-Train on Unlabeled Images!cited by 19
  20. 2024From GPT-4 to Gemini and Beyond: Assessing the Landscape of MLLMs on Generalizability, Trustworthiness and Causality through Four Modalitiescited by 19
  21. 2025FC-Attack: Jailbreaking Multimodal Large Language Models via Auto-Generated Flowchartscited by 18
  22. 2025SafeWork-R1: Coevolving Safety and Intelligence under the AI-45circ Lawcited by 14
  23. 2025Response Attack: Exploiting Contextual Priming to Jailbreak Large Language Modelscited by 10

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2026: 3.

3 papers.

  1. 2026SkillPyramid: A Hierarchical Skill Consolidation Framework for Self-Evolving Agentscited by 4
  2. 2026A Multimodal Automatic Redteaming Evaluation based on Atomic Jailbreak Strategy Decoupling and Combinationcited by 0
  3. 2026Multi2AV-Safety: Benchmarking Safety in Multimodal-to-Audio-Video Generationcited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2025Visual Contextual Attack: Jailbreaking MLLMs with Image-Driven Context Injection
  2. 2026Beyond Visual Safety: Jailbreaking Multimodal Large Language Models for Harmful Image Generation via Semantic-Agnostic Inputs
  3. 2026Jailbreaking Vision-Language Models Through the Visual Modality
  4. 2025VRSA: Jailbreaking Multimodal Large Language Models through Visual Reasoning Sequential Attack
  5. 2026Scenes as an Attack Surface: Uncovering Scene-Grounded Safety Failures in MLLMs
  6. 2026COMIC: Reference-Aware Safety Gating for Multimodal Large Language Models
  7. 2025Reimagining Safety Alignment with An Image
  8. 2026One Perturbation, Two Failure Modes: Probing VLM Safety via Embedding-Guided Typographic Perturbations