mlsanityData to 13 Oct 2026

Visual Contextual Attack: Jailbreaking MLLMs with Image-Driven Context Injection

arXiv 2507.02844 ↗First posted 3 Jul 2025cs.CV cs.CL cs.CRcited by 24

On the map: Multimodal LLMs / vision-language models → MLLM jailbreaks and adversarial robustness

also in 1 more
Abstract (arXiv)

With the emergence of strong vision language capabilities, multimodal large language models (MLLMs) have demonstrated tremendous potential for real-world applications. However, the security vulnerabilities exhibited by the visual modality pose significant challenges to deploying such models in open-world environments. Recent studies have successfully induced harmful responses from target MLLMs by encoding harmful textual semantics directly into visual inputs. However, in these approaches, the visual modality primarily serves as a trigger for unsafe behavior, often exhibiting semantic ambiguity and lacking grounding in realistic scenarios. In this work, we define a novel setting: vision-centric jailbreak, where visual information serves as a necessary component in constructing a complete and realistic jailbreak context. Building on this setting, we propose the VisCo (Visual Contextual) Attack. VisCo fabricates contextual dialogue using four distinct vision-focused strategies, dynamically generating auxiliary images when necessary to construct a vision-centric jailbreak scenario. To maximize attack effectiveness, it incorporates automatic toxicity obfuscation and semantic refinement to produce a final attack prompt that reliably triggers harmful responses from the target black-box MLLMs. Specifically, VisCo achieves a toxicity score of 4.78 and an Attack Success Rate (ASR) of 85% on MM-SafetyBench against GPT-4o, significantly outperforming the baseline, which achieves a toxicity score of 2.48 and an ASR of 22.2%. Code: https://github.com/Dtc7w3PQ/Visco-Attack.

Key facts. Cited by 24 papers in the map (data through 13 Oct 2026). Placed in MLLM jailbreaks and adversarial robustness, one of 16 directions in Multimodal LLMs / vision-language models. Builds on 24 of its 49 references that are in the map. First posted on arXiv 3 Jul 2025.

Builds on

24 of 49 references.

Closest ancestors · 7

Same direction, newest first

  1. 2025A Survey of Safety on Large Vision-Language Models: Attacks, Defenses and Evaluations14 Feb
  2. 2025Jailbreaking Multimodal Large Language Models via Shuffle Inconsistency9 Jan
  3. 2024VLSBench: Unveiling Visual Leakage in Multimodal Safety29 Nov
  4. 2024ETA: Evaluating Then Aligning Safety of Vision Language Models at Inference Time9 Oct
  5. 2024JailbreakZoo: Survey, Landscapes, and Horizons in Jailbreaking Large Language and Vision-Language Models26 Jun
  6. 2024MLLM-Protector: Ensuring MLLM's Safety without Hurting Performance5 Jan
  7. 2023FigStep: Jailbreaking Large Vision-Language Models via Typographic Visual Prompts9 Nov

Other references · 17

  1. 2023Improved Baselines with Visual Instruction Tuningcited by 4,389
  2. 2024Scaling Rectified Flow Transformers for High-Resolution Image Synthesiscited by 4,130
  3. 2023Fine-tuning Aligned Language Models Compromises Safety, Even When Users Do Not Intend To!cited by 1,141
  4. 2023Jailbreak and Guard Aligned Language Models with Only Few In-Context Demonstrationscited by 429
  5. 2024Great, Now Write an Article About That: The Crescendo Multi-Turn LLM Jailbreak Attackcited by 384
  6. 2024SALAD-Bench: A Hierarchical and Comprehensive Safety Benchmark for Large Language Modelscited by 228
  7. 2025X-Teaming: Multi-Turn Jailbreaks and Defenses with Adaptive Multi-Agentscited by 122
  8. 2023DRESS: Instructing Large Vision-Language Models to Align and Interact with Humans via Natural Language Feedbackcited by 103
  9. 2025H-CoT: Hijacking the Chain-of-Thought Safety Reasoning Mechanism to Jailbreak Large Reasoning Models, Including OpenAI o1/o3, DeepSeek-R1, and Gemini 2.0 Flash Thinkingcited by 97
  10. 2024Chain of Attack: a Semantic-Driven Contextual Multi-Turn attacker for LLMcited by 74
  11. 2024Speak Out of Turn: Safety Vulnerability of Large Language Models in Multi-turn Dialoguecited by 66
  12. 2025Safety in Large Reasoning Models: A Surveycited by 63
5 more
  1. 2023Bypassing the Safety Training of Open-Source LLMs with Priming Attackscited by 50
  2. 2025T2ISafety: Benchmark for Assessing Fairness, Toxicity, and Privacy in Image Generationcited by 48
  3. 2024The Better Angels of Machine Personality: How Personality Relates to LLM Safetycited by 21
  4. 2024From GPT-4 to Gemini and Beyond: Assessing the Landscape of MLLMs on Generalizability, Trustworthiness and Causality through Four Modalitiescited by 19
  5. 2025Dialogue Injection Attack: Jailbreaking LLMs through Context Manipulationcited by 14

Built on by

Cited by year (2026 to 13 Oct)

Citing papers per year: 2025: 7; 2026: 17.

24 papers.

Sort:

By area:

  1. 2025Layer-Aware Representation Filtering: Purifying Finetuning Data to Preserve LLM Safety Alignmentcited by 24
  2. 2025Response Attack: Exploiting Contextual Priming to Jailbreak Large Language Modelscited by 10
  3. 2025OmniSafeBench-MM: A Unified Benchmark and Toolbox for Multimodal Jailbreak Attack-Defense Evaluationcited by 10
  4. 2026SAVeS: Steering Safety Judgments in Vision-Language Models via Semantic Cuescited by 4
  5. 2026Visual Exclusivity Attacks: Automatic Multimodal Red Teaming via Agentic Planningcited by 4
  6. 2026Adversarial Attacks on Multimodal Large Language Models: A Comprehensive Surveycited by 4
  7. 2026TreeTeaming: Autonomous Red-Teaming of Vision-Language Models via Hierarchical Strategy Explorationcited by 3
  8. 2026Penny Wise, Pixel Foolish: Bypassing Price Constraints in Multimodal Agents via Visual Adversarial Perturbationscited by 3
  9. 2025Empowering Multimodal LLMs with External Tools: A Comprehensive Surveycited by 2
  10. 2025Self-adaptive Dataset Construction for Real-World Multimodal Safety Scenarioscited by 2
  11. 2026Red-teaming the Multimodal Reasoning: Jailbreaking Vision-Language Models via Cross-modal Entanglement Attackscited by 2
  12. 2026Visual Memory Injection Attacks for Multi-Turn Conversationscited by 2
12 more
  1. 2026Every Picture Tells a Dangerous Story: Memory-Augmented Multi-Agent Jailbreak Attacks on VLMscited by 2
  2. 2025LLaVAShield: Safeguarding Multimodal Multi-Turn Dialogues in Vision-Language Modelscited by 1
  3. 2025The Plot Twist: Jailbreaking Unified Multimodal Models with a Three-Act NarrativeAttackcited by 1
  4. 2026GAMBIT: A Gamified Jailbreak Framework for Multimodal Large Language Modelscited by 1
  5. 2026The Salami Slicing Threat: Exploiting Cumulative Risks in LLM Systemscited by 1
  6. 2026Dictionary-Aligned Concept Control for Safeguarding Multimodal LLMscited by 0
  7. 2026Seeing No Evil: Blinding Large Vision-Language Models to Safety Instructions via Adversarial Attention Hijackingcited by 0
  8. 2026New Wide-Net-Casting Jailbreak Attacks Risk Large Modelscited by 0
  9. 2026Modeling Complex Behaviors: Multi-Personality Composition and Dynamic Switching in Vision-Language Modelscited by 0
  10. 2026Security and Privacy in Retrieval-Augmented Generation: Architectures, Threats, Defenses, and Future Directions for Building Trustworthy Systemscited by 0
  11. 2026Fully Unleashing the Multimodal Attacker: Meta-Adaptive Jailbreaking of Vision-Language Modelscited by 0
  12. 2026Which Image Property Carries the Jailbreak? A Controlled Dissection of Image-to-Text Jailbreakscited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2026Beyond Visual Safety: Jailbreaking Multimodal Large Language Models for Harmful Image Generation via Semantic-Agnostic Inputs
  2. 2025VisCRA: A Visual Chain Reasoning Attack for Jailbreaking Multimodal Large Language Models
  3. 2025PiCo: Jailbreaking Multimodal Large Language Models via Pictorial Code Contextualization
  4. 2025Enhanced MLLM Black-Box Jailbreaking Attacks and Defenses
  5. 2026MLingualFC: Evaluating Jailbreak Vulnerabilities in Multilingual Vision-Language Models
  6. 2025Tit-for-Tat: Safeguarding Large Vision-Language Models Against Jailbreak Attacks via Adversarial Defense
  7. 2025Contextual Image Attack: How Visual Context Exposes Multimodal Safety Vulnerabilities
  8. 2024ImgTrojan: Jailbreaking Vision-Language Models with ONE Image