mlsanityData to 7 Oct 2026

Gradient-based Jailbreak Images for Multimodal Fusion Models

arXiv 2410.03489 ↗First posted 4 Oct 2024cs.CR cs.AIcited by 13

On the map: LLM safety and security → Vision-language and text-to-image safety

Abstract (arXiv)

Augmenting language models with image inputs may enable more effective jailbreak attacks through continuous optimization, unlike text inputs that require discrete optimization. However, new multimodal fusion models tokenize all input modalities using non-differentiable functions, which hinders straightforward attacks. In this work, we introduce the notion of a tokenizer shortcut that approximates tokenization with a continuous function and enables continuous optimization. We use tokenizer shortcuts to create the first end-to-end gradient image attacks against multimodal fusion models. We evaluate our attacks on Chameleon models and obtain jailbreak images that elicit harmful information for 72.5% of prompts. Jailbreak images outperform text jailbreaks optimized with the same objective and require 3x lower compute budget to optimize 50x more input tokens. Finally, we find that representation engineering defenses, like Circuit Breakers, trained only on text attacks can effectively transfer to adversarial image inputs.

Key facts. Cited by 13 papers in the map (data through 7 Oct 2026). Placed in Vision-language and text-to-image safety, one of 12 directions in LLM safety and security. Builds on 22 of its 25 references that are in the map. First posted on arXiv 4 Oct 2024.

Builds on

22 of 25 references.

Closest ancestors · 1

Same direction, newest first

  1. 2024Failures to Find Transferable Image Jailbreaks Between Vision-Language Models21 Jul

Other references · 21

  1. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,456
  2. 2023Visual Instruction Tuningcited by 7,792
  3. 2017Towards Deep Learning Models Resistant to Adversarial Attackscited by 6,326
  4. 2013Intriguing properties of neural networkscited by 5,114
  5. 2022Training a Helpful and Harmless Assistant with Reinforcement Learning from Human Feedbackcited by 3,634
  6. 2023Universal and Transferable Adversarial Attacks on Aligned Language Modelscited by 2,914
  7. 2023Jailbroken: How Does LLM Safety Training Fail?cited by 1,565
  8. 2018Obfuscated Gradients Give a False Sense of Security: Circumventing Defenses to Adversarial Examplescited by 1,259
  9. 2024Refusal in Language Models Is Mediated by a Single Directioncited by 960
  10. 2024Chameleon: Mixed-Modal Early-Fusion Foundation Modelscited by 904
  11. 2023AutoDAN: Generating Stealthy Jailbreak Prompts on Aligned Large Language Modelscited by 738
  12. 2023Baseline Defenses for Adversarial Attacks Against Aligned Language Modelscited by 608
9 more
  1. 2020On Adaptive Attacks to Adversarial Example Defensescited by 547
  2. 2017Adversarial Examples Are Not Easily Detected: Bypassing Ten Detection Methodscited by 542
  3. 2024JailbreakBench: An Open Robustness Benchmark for Jailbreaking Large Language Modelscited by 494
  4. 2023Jailbreaking ChatGPT via Prompt Engineering: An Empirical Studycited by 478
  5. 2024Prismatic VLMs: Investigating the Design Space of Visually-Conditioned Language Modelscited by 336
  6. 2023Are aligned neural networks adversarially aligned?cited by 305
  7. 2023MIMIC-IT: Multi-Modal In-Context Instruction Tuningcited by 264
  8. 2023Scalable and Transferable Black-Box Jailbreaks for Language Models via Persona Modulationcited by 202
  9. 2024FLRT: Fluent Student-Teacher Redteamingcited by 9

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2024: 2; 2025: 7; 2026: 4.

13 papers.

Sort:

By area:

  1. 2024Llama Guard 3 Vision: Safeguarding Human-AI Image Understanding Conversationscited by 118
  2. 2025OverThink: Slowdown Attacks on Reasoning LLMscited by 94
  3. 2024Best-of-N Jailbreakingcited by 71
  4. 2025Position: Adversarial ML for LLMs Is Not Making Any Progresscited by 14
  5. 2025MIP against Agent: Malicious Image Patches Hijacking Multimodal OS Agentscited by 12
  6. 2025Trojan Horse Prompting: Jailbreaking Conversational Multimodal Models by Forging Assistant Messagecited by 1
  7. 2025Differences That Matter: Auditing Models for Capability Gap Discovery and Rectificationcited by 1
  8. 2025Multi-Agent Framework for Threat Mitigation and Resilience in AI-Based Systemscited by 1
  9. 2026Adversarial Diffusion Across Modalities: A Fusion Survey of Attacks, Defenses, and Evaluation for Text, Vision, and Vision-Language Modelscited by 1
  10. 2025Understanding Adversarial Transfer: Why Representation-Space Attacks Fail Where Data-Space Attacks Succeedcited by 0
  11. 2026Laundering AI Authority with Adversarial Examplescited by 0
  12. 2026Break the Brake, Not the Wheel: Untargeted Jailbreak via Entropy Maximizationcited by 0
1 more
  1. 2026StructBreak: Structural Cognitive Overload-Induced Safety Failures in MLLMscited by 0