mlsanityData to 7 Oct 2026

Jailbreaking Multimodal Large Language Models via Shuffle Inconsistency

arXiv 2501.04931 ↗First posted 9 Jan 2025cs.CR cs.AI cs.CLcited by 42

On the map: Multimodal LLMs / vision-language models → MLLM jailbreaks and adversarial robustness

also in 1 more
Abstract (arXiv)

Multimodal Large Language Models (MLLMs) have achieved impressive performance and have been put into practical use in commercial applications, but they still have potential safety mechanism vulnerabilities. Jailbreak attacks are red teaming methods that aim to bypass safety mechanisms and discover MLLMs' potential risks. Existing MLLMs' jailbreak methods often bypass the model's safety mechanism through complex optimization methods or carefully designed image and text prompts. Despite achieving some progress, they have a low attack success rate on commercial closed-source MLLMs. Unlike previous research, we empirically find that there exists a Shuffle Inconsistency between MLLMs' comprehension ability and safety ability for the shuffled harmful instruction. That is, from the perspective of comprehension ability, MLLMs can understand the shuffled harmful text-image instructions well. However, they can be easily bypassed by the shuffled harmful instructions from the perspective of safety ability, leading to harmful responses. Then we innovatively propose a text-image jailbreak attack named SI-Attack. Specifically, to fully utilize the Shuffle Inconsistency and overcome the shuffle randomness, we apply a query-based black-box optimization method to select the most harmful shuffled inputs based on the feedback of the toxic judge model. A series of experiments show that SI-Attack can improve the attack's performance on three benchmarks. In particular, SI-Attack can obviously improve the attack success rate for commercial MLLMs such as GPT-4o or Claude-3.5-Sonnet.

Key facts. Cited by 42 papers in the map (data through 7 Oct 2026). Placed in MLLM jailbreaks and adversarial robustness, one of 16 directions in Multimodal LLMs / vision-language models. Builds on 32 of its 40 references that are in the map. First posted on arXiv 9 Jan 2025.

Builds on

32 of 40 references.

Closest ancestors · 7

Same direction, newest first

  1. 2024Cross-Modal Safety Alignment: Is textual unlearning all you need?27 May
  2. 2024Images are Achilles' Heel of Alignment: Exploiting Visual Vulnerabilities for Jailbreaking Multimodal Large Language Models14 Mar
  3. 2023MM-SafetyBench: A Benchmark for Safety Evaluation of Multimodal Large Language Models29 Nov
  4. 2023FigStep: Jailbreaking Large Vision-Language Models via Typographic Visual Prompts9 Nov
  5. 2023Jailbreak in pieces: Compositional Adversarial Attacks on Multi-Modal Language Models26 Jul
  6. 2023Visual Adversarial Examples Jailbreak Aligned Large Language Models22 Jun
  7. 2023On Evaluating Adversarial Robustness of Large Vision-Language Models26 May

Other references · 25

  1. 2024The Llama 3 Herd of Modelscited by 14,192
  2. 2017Towards Deep Learning Models Resistant to Adversarial Attackscited by 6,326
  3. 2023Universal and Transferable Adversarial Attacks on Aligned Language Modelscited by 2,914
  4. 2023MiniGPT-4: Enhancing Vision-Language Understanding with Advanced Large Language Modelscited by 2,689
  5. 2023InternVL: Scaling up Vision Foundation Models and Aligning for Generic Visual-Linguistic Taskscited by 2,534
  6. 2023G-Eval: NLG Evaluation using GPT-4 with Better Human Alignmentcited by 2,205
  7. 2023Qwen-VL: A Versatile Vision-Language Model for Understanding, Localization, Text Reading, and Beyondcited by 2,179
  8. 2023Jailbroken: How Does LLM Safety Training Fail?cited by 1,565
  9. 2023Llama Guard: LLM-based Input-Output Safeguard for Human-AI Conversationscited by 1,124
  10. 2023AutoDAN: Generating Stealthy Jailbreak Prompts on Aligned Large Language Modelscited by 738
  11. 2023Baseline Defenses for Adversarial Attacks Against Aligned Language Modelscited by 608
  12. 2022When and why vision-language models behave like bags-of-words, and what to do about it?cited by 603
13 more
  1. 2024LLaVA-NeXT-Interleave: Tackling Multi-image, Video, and 3D in Large Multimodal Modelscited by 571
  2. 2023"Do Anything Now": Characterizing and Evaluating In-The-Wild Jailbreak Prompts on Large Language Modelscited by 457
  3. 2023Detecting Language Model Attacks with Perplexitycited by 411
  4. 2022Red-Teaming the Stable Diffusion Safety Filtercited by 257
  5. 2023Multilingual Jailbreak Challenges in Large Language Modelscited by 222
  6. 2023MMA-Diffusion: MultiModal Attack on Diffusion Modelscited by 184
  7. 2023Image Hijacks: Adversarial Images can Control Generative Models at Runtimecited by 165
  8. 2024Safety Fine-Tuning at (Almost) No Cost: A Baseline for Vision Large Language Modelscited by 160
  9. 2024Jailbreaking Attack against Multimodal Large Language Modelcited by 128
  10. 2024On Prompt-Driven Safeguarding for Large Language Modelscited by 109
  11. 2024Unveiling the Safety of GPT-4o: An Empirical Study using Jailbreak Attackscited by 35
  12. 2024MRJ-Agent: An Effective Jailbreak Agent for Multi-Round Dialoguecited by 28
  13. 2024The First to Know: How Token Distributions Reveal Hidden Knowledge in Large Vision-Language Models?cited by 24

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2025: 22; 2026: 20.

42 papers.

Sort:

By area:

  1. 2025Visual Contextual Attack: Jailbreaking MLLMs with Image-Driven Context Injectioncited by 23
  2. 2025FC-Attack: Jailbreaking Multimodal Large Language Models via Auto-Generated Flowchartscited by 15
  3. 2025From LLMs to MLLMs to Agents: A Survey of Emerging Paradigms in Jailbreak Attacks and Defenses within LLM Ecosystemcited by 11
  4. 2025FORCE: Transferable Visual Jailbreaking Attacks via Feature Over-Reliance CorrEctioncited by 11
  5. 2025JailDAM: Jailbreak Detection with Adaptive Memory for Vision-Language Modelcited by 10
  6. 2025USB: A Comprehensive and Unified Safety Evaluation Benchmark for Multimodal Large Language Modelscited by 10
  7. 2025OmniSafeBench-MM: A Unified Benchmark and Toolbox for Multimodal Jailbreak Attack-Defense Evaluationcited by 10
  8. 2025Detoxifying Large Language Models via Autoregressive Reward Guided Representation Editingcited by 9
  9. 2025Implicit Jailbreak Attacks via Cross-Modal Information Concealment on Vision-Language Modelscited by 5
  10. 2025AMIA: Automatic Masking and Joint Intention Analysis Makes LVLMs Robust Jailbreak Defenderscited by 5
  11. 2025Mind the Web: The Security of Web Use Agentscited by 5
  12. 2025Can MLLMs Read the Room? A Multimodal Benchmark for Assessing Deception in Multi-Party Social Interactionscited by 5
30 more
  1. 2026Evolving Contextual Safety in Multi-Modal Large Language Models via Inference-Time Self-Reflective Memorycited by 5
  2. 2026Visual Exclusivity Attacks: Automatic Multimodal Red Teaming via Agentic Planningcited by 4
  3. 2025PRISM: Programmatic Reasoning with Image Sequence Manipulation for LVLM Jailbreakingcited by 3
  4. 2025ARMs: Adaptive Red-Teaming Agent against Multimodal Models with Plug-and-Play Attackscited by 3
  5. 2025JPRO: Automated Multimodal Jailbreaking via Multi-Agent Collaboration Frameworkcited by 3
  6. 2026Making MLLMs Blind: Adversarial Smuggling Attacks in MLLM Content Moderationcited by 3
  7. 2025Self-Aware Safety Augmentation: Leveraging Internal Semantic Understanding to Enhance Safety in Vision-Language Modelscited by 2
  8. 2025Active Honeypot Guardrail System: Probing and Confirming Multi-Turn LLM Jailbreakscited by 2
  9. 2025Contextual Image Attack: How Visual Context Exposes Multimodal Safety Vulnerabilitiescited by 2
  10. 2025VRSA: Jailbreaking Multimodal Large Language Models through Visual Reasoning Sequential Attackcited by 2
  11. 2026Multi-turn Jailbreaking Attack in Multi-Modal Large Language Modelscited by 2
  12. 2026TreeTeaming: Autonomous Red-Teaming of Vision-Language Models via Hierarchical Strategy Explorationcited by 2
  13. 2025Trojan Horse Prompting: Jailbreaking Conversational Multimodal Models by Forging Assistant Messagecited by 1
  14. 2025Jailbreaking Large Vision Language Models in Intelligent Transportation Systemscited by 1
  15. 2026GAMBIT: A Gamified Jailbreak Framework for Multimodal Large Language Modelscited by 1
  16. 2026MIDAS: Multi-Image Dispersion and Semantic Reconstruction for Jailbreaking MLLMscited by 1
  17. 2026Improving Safety Alignment via Balanced Direct Preference Optimizationcited by 1
  18. 2026The Salami Slicing Threat: Exploiting Cumulative Risks in LLM Systemscited by 1
  19. 2025VLMs Can Aggregate Scattered Training Patchescited by 0
  20. 2026Beyond Visual Safety: Jailbreaking Multimodal Large Language Models for Harmful Image Generation via Semantic-Agnostic Inputscited by 0
  21. 2026Stage-wise Attention-Guided Region Sequencing for Adversarial Attacks on Large Vision-Language Modelscited by 0
  22. 2026Two Birds, One Projection: Harmonizing Safety and Utility in LVLMs via Inference-time Feature Projectioncited by 0
  23. 2026Mosaic: Multimodal Jailbreak against Closed-Source VLMs via Multi-View Ensemble Optimizationcited by 0
  24. 2026Conceal, Reconstruct, Jailbreak: Exploiting the Reconstruction-Concealment Tradeoff in MLLMscited by 0
  25. 2026Jailbreaking Multimodal Large Language Models using Multi-Clip Videocited by 0
  26. 2026Harnessing Textual Refusal Directions for Multimodal Safetycited by 0
  27. 2026Overloading Large Vision-Language Models for Jailbreakingcited by 0
  28. 2026Adversarial Style Optimization: Enhancing VLM Jailbreaks by GRPO-based Stylistic Triggers Optimizationcited by 0
  29. 2026Guardrail-Agnostic Societal Bias Evaluation in Large Vision-Language Modelscited by 0
  30. 2026UnifiedAttack: Evaluating the Safety of Large Multimodal Models in Synergistic Harmful Image-Text Generationcited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2024JailBreakV: A Benchmark for Assessing the Robustness of MultiModal Large Language Models against Jailbreak Attacks
  2. 2025Align is not Enough: Multimodal Universal Jailbreak Attack against Multimodal Large Language Models
  3. 2024textitMMJ-Bench: A Comprehensive Study on Jailbreak Attacks and Defenses for Multimodal Large Language Models
  4. 2025JPS: Jailbreak Multimodal Large Language Models with Collaborative Visual Perturbation and Textual Steering
  5. 2026DMN: A Compositional Framework for Jailbreaking Multimodal LLMs with Multi-Image Inputs
  6. 2025Probabilistic Modeling of Jailbreak on Multimodal LLMs: From Quantification to Application
  7. 2025PolyJailbreak: Cross-Modal Jailbreaking Attacks on Black-Box Multimodal LLMs
  8. 2024Divide and Conquer: A Hybrid Strategy Defeats Multimodal Large Language Models