mlsanityData to 7 Oct 2026

From LLMs to MLLMs: Exploring the Landscape of Multimodal Jailbreaking

arXiv 2406.14859 ↗First posted 21 Jun 2024cs.CL cs.AIcited by 22

On the map: Multimodal LLMs / vision-language models → MLLM jailbreaks and adversarial robustness

also in 1 more
Abstract (arXiv)

The rapid development of Large Language Models (LLMs) and Multimodal Large Language Models (MLLMs) has exposed vulnerabilities to various adversarial attacks. This paper provides a comprehensive overview of jailbreaking research targeting both LLMs and MLLMs, highlighting recent advancements in evaluation benchmarks, attack techniques and defense strategies. Compared to the more advanced state of unimodal jailbreaking, multimodal domain remains underexplored. We summarize the limitations and potential research directions of multimodal jailbreaking, aiming to inspire future research and further enhance the robustness and security of MLLMs.

Key facts. Cited by 22 papers in the map (data through 7 Oct 2026). Placed in MLLM jailbreaks and adversarial robustness, one of 16 directions in Multimodal LLMs / vision-language models. Builds on 51 of its 81 references that are in the map. First posted on arXiv 21 Jun 2024.

Builds on

51 of 81 references.

Closest ancestors · 5

Same direction, newest first

  1. 2024JailBreakV: A Benchmark for Assessing the Robustness of MultiModal Large Language Models against Jailbreak Attacks3 Apr
  2. 2024Eyes Closed, Safety On: Protecting Multimodal LLMs via Image-to-Text Transformation14 Mar
  3. 2024Agent Smith: A Single Image Can Jailbreak One Million Multimodal LLM Agents Exponentially Fast13 Feb
  4. 2023FigStep: Jailbreaking Large Vision-Language Models via Typographic Visual Prompts9 Nov
  5. 2023Jailbreak in pieces: Compositional Adversarial Attacks on Multi-Modal Language Models26 Jul

Other references · 46

  1. 2021High-Resolution Image Synthesis with Latent Diffusion Modelscited by 18,694
  2. 2023LLaMA: Open and Efficient Foundation Language Modelscited by 13,958
  3. 2019DistilBERT, a distilled version of BERT: smaller, faster, cheaper and lightercited by 4,243
  4. 2022Constitutional AI: Harmlessness from AI Feedbackcited by 2,950
  5. 2021TruthfulQA: Measuring How Models Mimic Human Falsehoodscited by 2,919
  6. 2023Jailbroken: How Does LLM Safety Training Fail?cited by 1,565
  7. 2023Jailbreaking Black Box Large Language Models in Twenty Queriescited by 1,326
  8. 2023Fine-tuning Aligned Language Models Compromises Safety, Even When Users Do Not Intend To!cited by 1,106
  9. 2023GPTFUZZER: Red Teaming Large Language Models with Auto-Generated Jailbreak Promptscited by 520
  10. 2023DecodingTrust: A Comprehensive Assessment of Trustworthiness in GPT Modelscited by 505
  11. 2023Jailbreaking ChatGPT via Prompt Engineering: An Empirical Studycited by 478
  12. 2023Catastrophic Jailbreak of Open-source LLMs via Exploiting Generationcited by 439
34 more
  1. 2023SmoothLLM: Defending Large Language Models Against Jailbreaking Attackscited by 434
  2. 2023Detecting Language Model Attacks with Perplexitycited by 411
  3. 2023GPT-4 Is Too Smart To Be Safe: Stealthy Chat with LLMs via Ciphercited by 401
  4. 2023Jailbreak and Guard Aligned Language Models with Only Few In-Context Demonstrationscited by 401
  5. 2023Trustworthy LLMs: a Survey and Guideline for Evaluating Large Language Models' Alignmentcited by 394
  6. 2023Multi-step Jailbreaking Privacy Attacks on ChatGPTcited by 320
  7. 2023Low-Resource Languages Jailbreak GPT-4cited by 290
  8. 2023Shadow Alignment: The Ease of Subverting Safely-Aligned Language Modelscited by 284
  9. 2023Exploiting Programmatic Behavior of LLMs: Dual-Use Through Standard Security Attackscited by 268
  10. 2023Certifying LLM Safety against Adversarial Promptingcited by 248
  11. 2024SafeDecoding: Defending against Jailbreak Attacks via Safety-Aware Decodingcited by 233
  12. 2023LLM Self Defense: By Self Examination, LLMs Know They Are Being Trickedcited by 226
  13. 2023PromptRobust: Towards Evaluating the Robustness of Large Language Models on Adversarial Promptscited by 218
  14. 2023Red-Teaming Large Language Models using Chain of Utterances for Safety-Alignmentcited by 213
  15. 2023Scalable and Transferable Black-Box Jailbreaks for Language Models via Persona Modulationcited by 202
  16. 2023LoRA Fine-tuning Efficiently Undoes Safety Training in Llama 2-Chat 70Bcited by 183
  17. 2023Do-Not-Answer: A Dataset for Evaluating Safeguards in LLMscited by 176
  18. 2023Image Hijacks: Adversarial Images can Control Generative Models at Runtimecited by 165
  19. 2024Robust Prompt Optimization for Defending Language Models Against Jailbreaking Attackscited by 150
  20. 2023RAIN: Your Language Models Can Align Themselves without Finetuningcited by 149
  21. 2023From ChatGPT to ThreatGPT: Impact of Generative AI in Cybersecurity and Privacycited by 137
  22. 2023Universal Jailbreak Backdoors from Poisoned Human Feedbackcited by 120
  23. 2024Making Them Ask and Answer: Jailbreaking Large Language Models in Few Queries via Disguise and Reconstructioncited by 95
  24. 2023CValues: Measuring the Values of Chinese Large Language Models from Safety to Responsibilitycited by 80
  25. 2023Cognitive Overload: Jailbreaking Large Language Models with Overloaded Logical Thinkingcited by 78
  26. 2024Defending Large Language Models against Jailbreak Attacks via Semantic Smoothingcited by 68
  27. 2024Speak Out of Turn: Safety Vulnerability of Large Language Models in Multi-turn Dialoguecited by 64
  28. 2024Pandora: Jailbreak GPTs by Retrieval Augmented Generation Poisoningcited by 62
  29. 2023Latent Jailbreak: A Benchmark for Evaluating Text Safety and Output Robustness of Large Language Modelscited by 56
  30. 2024Distract Large Language Models for Automatic Jailbreak Attackcited by 40
  31. 2024Red Teaming GPT-4V: Are GPT-4V Safe Against Uni/Multi-Modal Jailbreak Attacks?cited by 32
  32. 2023ToViLaG: Your Visual-Language Generative Model is Also An Evildoercited by 27
  33. 2024Break the Breakout: Reinventing LM Defense Against Jailbreak Attacks with Self-Refinementcited by 25
  34. 2023FFT: Towards Harmlessness Evaluation and Analysis for LLMs with Factuality, Fairness, Toxicitycited by 22

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2024: 6; 2025: 10; 2026: 6.

22 papers.

Sort:

By area:

  1. 2024A Survey of Attacks on Large Vision-Language Models: Resources, Advances, and Future Trendscited by 82
  2. 2025A Survey of Safety on Large Vision-Language Models: Attacks, Defenses and Evaluationscited by 66
  3. 2024Core Knowledge Deficits in Multi-Modal Language Modelscited by 38
  4. 2024Adversarial Attacks of Vision Tasks in the Past 10 Years: A Surveycited by 22
  5. 2024Surveying the MLLM Landscape: A Meta-Review of Current Surveyscited by 20
  6. 2025Jailbreak-AudioBench: In-Depth Evaluation and Analysis of Jailbreak Threats for Large Audio Language Modelscited by 17
  7. 2024Manipulation Facing Threats: Evaluating Physical Vulnerabilities in End-to-End Vision Language Action Modelscited by 16
  8. 2025From LLMs to MLLMs to Agents: A Survey of Emerging Paradigms in Jailbreak Attacks and Defenses within LLM Ecosystemcited by 11
  9. 2025Exploring Jailbreak Attacks on LLMs through Intent Concealment and Diversioncited by 9
  10. 2025AdPO: Enhancing the Adversarial Robustness of Large Vision-Language Models with Preference Optimizationcited by 8
  11. 2026Evolving Contextual Safety in Multi-Modal Large Language Models via Inference-Time Self-Reflective Memorycited by 5
  12. 2025SmoothGuard: Defending Multimodal Large Language Models with Noise Perturbation and Clustering Aggregationcited by 2
10 more
  1. 2024Divide and Conquer: A Hybrid Strategy Defeats Multimodal Large Language Modelscited by 1
  2. 2025How Jailbreak Defenses Work and Ensemble? A Mechanistic Investigationcited by 1
  3. 2025Dynamic Token Reweighting for Robust Vision-Language Modelscited by 1
  4. 2025Layer-Wise Perturbations via Sparse Autoencoders for Adversarial Text Generationcited by 1
  5. 2025LLM in the Middle: A Systematic Review of Threats and Mitigations to Real-World LLM-based Systemscited by 1
  6. 2026GAMBIT: A Gamified Jailbreak Framework for Multimodal Large Language Modelscited by 1
  7. 2026Furina: Fragmented Uncertainty-Driven Refusal Instability Attackcited by 1
  8. 2026Questionnaire Responses Do not Capture the Safety of AI Agentscited by 0
  9. 2026MLingualFC: Evaluating Jailbreak Vulnerabilities in Multilingual Vision-Language Modelscited by 0
  10. 2026Evolving Safety Landscape of Multi-modal Large Language Models: A Survey of Emerging Threats and Safeguardscited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2025Probabilistic Modeling of Jailbreak on Multimodal LLMs: From Quantification to Application
  2. 2024textitMMJ-Bench: A Comprehensive Study on Jailbreak Attacks and Defenses for Multimodal Large Language Models
  3. 2025Align is not Enough: Multimodal Universal Jailbreak Attack against Multimodal Large Language Models
  4. 2024Efficient LLM-Jailbreaking via Multimodal-LLM Jailbreak
  5. 2024Jailbreak Attacks and Defenses Against Large Language Models: A Survey
  6. 2025Do Internal Layers of LLMs Reveal Patterns for Jailbreak Detection?
  7. 2026DMN: A Compositional Framework for Jailbreaking Multimodal LLMs with Multi-Image Inputs
  8. 2025The Tower of Babel Revisited: Multilingual Jailbreak Prompts on Closed-Source Large Language Models