mlsanityData to 7 Oct 2026

Multi-turn Jailbreaking Attack in Multi-Modal Large Language Models

arXiv 2601.05339 ↗First posted 8 Jan 2026cs.CR cs.AIcited by 2

On the map: LLM safety and security → Vision-language and text-to-image safety

Abstract (arXiv)

In recent years, the security vulnerabilities of Multi-modal Large Language Models (MLLMs) have become a serious concern in the Generative Artificial Intelligence (GenAI) research. These highly intelligent models, capable of performing multi-modal tasks with high accuracy, are also severely susceptible to carefully launched security attacks, such as jailbreaking attacks, which can manipulate model behavior and bypass safety constraints. This paper introduces MJAD-MLLMs, a holistic framework that systematically analyzes the proposed Multi-turn Jailbreaking Attacks and multi-LLM-based defense techniques for MLLMs. In this paper, we make three original contributions. First, we introduce a novel multi-turn jailbreaking attack to exploit the vulnerabilities of the MLLMs under multi-turn prompting. Second, we propose a novel fragment-optimized and multi-LLM defense mechanism, called FragGuard, to effectively mitigate jailbreaking attacks in the MLLMs. Third, we evaluate the efficacy of the proposed attacks and defenses through extensive experiments on several state-of-the-art (SOTA) open-source and closed-source MLLMs and benchmark datasets, and compare their performance with the existing techniques.

Key facts. Cited by 2 papers in the map (data through 7 Oct 2026). Placed in Vision-language and text-to-image safety, one of 12 directions in LLM safety and security. Builds on 47 of its 56 references that are in the map. First posted on arXiv 8 Jan 2026.

Builds on

47 of 56 references.

Closest ancestors · 8

Same direction, newest first

  1. 2025Jailbreaking Multimodal Large Language Models via Shuffle Inconsistency9 Jan
  2. 2024Jailbreak Attacks and Defenses against Multimodal Generative Models: A Survey14 Nov
  3. 2024IDEATOR: Jailbreaking and Benchmarking Large Vision-Language Models Using Themselves29 Oct
  4. 2024A Survey of Attacks on Large Vision-Language Models: Resources, Advances, and Future Trends10 Jul
  5. 2024Images are Achilles' Heel of Alignment: Exploiting Visual Vulnerabilities for Jailbreaking Multimodal Large Language Models14 Mar
  6. 2024AdaShield: Safeguarding Multimodal Large Language Models from Structure-based Attack via Adaptive Shield Prompting14 Mar
  7. 2024ArtPrompt: ASCII Art-based Jailbreak Attacks against Aligned LLMs19 Feb
  8. 2023FigStep: Jailbreaking Large Vision-Language Models via Typographic Visual Prompts9 Nov

Other references · 39

  1. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,456
  2. 2024The Llama 3 Herd of Modelscited by 14,192
  3. 2025Qwen3 Technical Reportcited by 8,195
  4. 2023Visual Instruction Tuningcited by 7,792
  5. 2025Qwen2.5-VL Technical Reportcited by 5,418
  6. 2024GPT-4o System Cardcited by 4,467
  7. 2023Improved Baselines with Visual Instruction Tuningcited by 4,362
  8. 2025Gemini 2.5: Pushing the Frontier with Advanced Reasoning, Multimodality, Long Context, and Next Generation Agentic Capabilitiescited by 3,655
  9. 2023MiniGPT-4: Enhancing Vision-Language Understanding with Advanced Large Language Modelscited by 2,689
  10. 2023Vision-Language Models for Vision Tasks: A Surveycited by 834
  11. 2023AutoDAN: Generating Stealthy Jailbreak Prompts on Aligned Large Language Modelscited by 738
  12. 2024LLaVA-NeXT-Interleave: Tackling Multi-image, Video, and 3D in Large Multimodal Modelscited by 571
27 more
  1. 2024Jailbreaking Leading Safety-Aligned LLMs with Simple Adaptive Attackscited by 450
  2. 2024Great, Now Write an Article About That: The Crescendo Multi-Turn LLM Jailbreak Attackcited by 364
  3. 2023AudioPaLM: A Large Language Model That Can Speak and Listencited by 333
  4. 2023MM-SafetyBench: A Benchmark for Safety Evaluation of Multimodal Large Language Modelscited by 255
  5. 2024Jailbreak Attacks and Defenses Against Large Language Models: A Surveycited by 215
  6. 2024Audio Flamingo: A Novel Audio Language Model with Few-Shot Learning and Dialogue Abilitiescited by 183
  7. 2024Security and Privacy Challenges of Large Language Models: A Surveycited by 179
  8. 2024AutoDefense: Multi-Agent LLM Defense against Jailbreak Attackscited by 135
  9. 2024Eyes Closed, Safety On: Protecting Multimodal LLMs via Image-to-Text Transformationcited by 114
  10. 2024MLLM-Protector: Ensuring MLLM's Safety without Hurting Performancecited by 111
  11. 2023Vision-Language Models in Remote Sensing: Current Progress and Future Trendscited by 93
  12. 2024LLMs know their vulnerabilities: Uncover Safety Gaps through Natural Distribution Shiftscited by 80
  13. 2024Defending Large Language Models Against Jailbreak Attacks via Layer-specific Editingcited by 77
  14. 2024Chain of Attack: a Semantic-Driven Contextual Multi-Turn attacker for LLMcited by 73
  15. 2024A Comprehensive Review of Multimodal Large Language Models: Performance and Challenges Across Different Taskscited by 57
  16. 2024Multi-Turn Context Jailbreak Attack on Large Language Models From First Principlescited by 54
  17. 2025Foot-In-The-Door: A Multi-turn Jailbreak for LLMscited by 50
  18. 2024Bag of Tricks: Benchmarking of Jailbreak Attacks on LLMscited by 47
  19. 2025JBShield: Defending Large Language Models from Jailbreak Attacks through Activated Concept Analysis and Manipulationcited by 46
  20. 2024MRJ-Agent: An Effective Jailbreak Agent for Multi-Round Dialoguecited by 28
  21. 2024CE-CoLLM: Efficient and Adaptive Large Language Models Through Cloud-Edge Collaborationcited by 25
  22. 2025BadToken: Token-level Backdoor Attacks to Multi-modal Large Language Modelscited by 15
  23. 2025X-Guard: Multilingual Guard Agent for Content Moderationcited by 13
  24. 2025Distributed LLMs and Multimodal Large Language Models: A Survey on Advances, Challenges, and Future Directionscited by 10
  25. 2025System Prompt Extraction Attacks and Defenses in Large Language Modelscited by 6
  26. 2025CEQuest: Benchmarking Large Language Models for Construction Estimationcited by 6
  27. 2025SafeMT: Multi-turn Safety for Multimodal Language Modelscited by 4

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2026: 2.

2 papers.

  1. 2026Visual Memory Injection Attacks for Multi-Turn Conversationscited by 2
  2. 2026Surviving the Unseen: Predictive Defense for Novel Multi-Turn Multimodal Attackscited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2024textitMMJ-Bench: A Comprehensive Study on Jailbreak Attacks and Defenses for Multimodal Large Language Models
  2. 2024From LLMs to MLLMs: Exploring the Landscape of Multimodal Jailbreaking
  3. 2024Jailbreaking Attack against Multimodal Large Language Model
  4. 2024JailBreakV: A Benchmark for Assessing the Robustness of MultiModal Large Language Models against Jailbreak Attacks
  5. 2024Divide and Conquer: A Hybrid Strategy Defeats Multimodal Large Language Models
  6. 2024Heuristic-Induced Multimodal Risk Distribution Jailbreak Attack for Multimodal Large Language Models
  7. 2025From LLMs to MLLMs to Agents: A Survey of Emerging Paradigms in Jailbreak Attacks and Defenses within LLM Ecosystem
  8. 2025Probabilistic Modeling of Jailbreak on Multimodal LLMs: From Quantification to Application