# IDEATOR: Jailbreaking and Benchmarking Large Vision-Language Models Using Themselves

arXiv: https://arxiv.org/abs/2411.00827 · HTML: https://mlsanity.com/paper/2411.00827/

## Key facts

- Cited by 37 papers in the map (data through 14 Oct 2026).
- Placed in MLLM jailbreaks and adversarial robustness, one of 16 directions in Multimodal LLMs / vision-language models.
- Builds on 37 of its 44 references that are in the map.
- First posted on arXiv 29 Oct 2024.

## Placed in

- Multimodal LLMs / vision-language models / [MLLM jailbreaks and adversarial robustness](https://mlsanity.com/mllm/mllm-jailbreaks-and-adversarial-robustness/)
- LLM safety and security / [Vision-language and text-to-image safety](https://mlsanity.com/safety/vision-language-and-text-to-image-safety/)

## Builds on (most cited first, 30 of 37)

- [Learning Transferable Visual Models From Natural Language Supervision](https://mlsanity.com/paper/2103.00020/) (2021)
- [Language Models are Few-Shot Learners](https://mlsanity.com/paper/2005.14165/) (2020)
- [An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale](https://mlsanity.com/paper/2010.11929/) (2020)
- [High-Resolution Image Synthesis with Latent Diffusion Models](https://mlsanity.com/paper/2112.10752/) (2021)
- [GPT-4 Technical Report](https://mlsanity.com/paper/2303.08774/) (2023)
- [The Llama 3 Herd of Models](https://mlsanity.com/paper/2407.21783/) (2024)
- [LLaMA: Open and Efficient Foundation Language Models](https://mlsanity.com/paper/2302.13971/) (2023)
- [Chain-of-Thought Prompting Elicits Reasoning in Large Language Models](https://mlsanity.com/paper/2201.11903/) (2022)
- [Visual Instruction Tuning](https://mlsanity.com/paper/2304.08485/) (2023)
- [Qwen2-VL: Enhancing Vision-Language Model's Perception of the World at Any Resolution](https://mlsanity.com/paper/2409.12191/) (2024)
- [Gemini 1.5: Unlocking multimodal understanding across millions of tokens of context](https://mlsanity.com/paper/2403.05530/) (2024)
- [Universal and Transferable Adversarial Attacks on Aligned Language Models](https://mlsanity.com/paper/2307.15043/) (2023)
- [InstructBLIP: Towards General-purpose Vision-Language Models with Instruction Tuning](https://mlsanity.com/paper/2305.06500/) (2023)
- [LLaVA-OneVision: Easy Visual Task Transfer](https://mlsanity.com/paper/2408.03326/) (2024)
- [MiniGPT-4: Enhancing Vision-Language Understanding with Advanced Large Language Models](https://mlsanity.com/paper/2304.10592/) (2023)
- [Jailbreaking Black Box Large Language Models in Twenty Queries](https://mlsanity.com/paper/2310.08419/) (2023)
- [Chameleon: Mixed-Modal Early-Fusion Foundation Models](https://mlsanity.com/paper/2405.09818/) (2024)
- [MiniGPT-v2: large language model as a unified interface for vision-language multi-task learning](https://mlsanity.com/paper/2310.09478/) (2023)
- [Make-A-Scene: Scene-Based Text-to-Image Generation with Human Priors](https://mlsanity.com/paper/2203.13131/) (2022)
- [Diffusion Models for Adversarial Purification](https://mlsanity.com/paper/2205.07460/) (2022)
- [FigStep: Jailbreaking Large Vision-Language Models via Typographic Visual Prompts](https://mlsanity.com/paper/2311.05608/) (2023)
- [Visual Adversarial Examples Jailbreak Aligned Large Language Models](https://mlsanity.com/paper/2306.13213/) (2023)
- [Are aligned neural networks adversarially aligned?](https://mlsanity.com/paper/2306.15447/) (2023)
- [MM-SafetyBench: A Benchmark for Safety Evaluation of Multimodal Large Language Models](https://mlsanity.com/paper/2311.17600/) (2023)
- [Jailbreak in pieces: Compositional Adversarial Attacks on Multi-Modal Language Models](https://mlsanity.com/paper/2307.14539/) (2023)
- [JailBreakV: A Benchmark for Assessing the Robustness of MultiModal Large Language Models against Jailbreak Attacks](https://mlsanity.com/paper/2404.03027/) (2024)
- [Survey of Vulnerabilities in Large Language Models Revealed by Adversarial Attacks](https://mlsanity.com/paper/2310.10844/) (2023)
- [Image Hijacks: Adversarial Images can Control Generative Models at Runtime](https://mlsanity.com/paper/2309.00236/) (2023)
- [Jailbreaking Attack against Multimodal Large Language Model](https://mlsanity.com/paper/2402.02309/) (2024)
- [AdaShield: Safeguarding Multimodal Large Language Models from Structure-based Attack via Adaptive Shield Prompting](https://mlsanity.com/paper/2403.09513/) (2024)

## Built on by (most cited first, 30 of 37)

- [A Survey of Safety on Large Vision-Language Models: Attacks, Defenses and Evaluations](https://mlsanity.com/paper/2502.14881/) (2025)
- [VPI-Bench: Visual Prompt Injection Attacks for Computer-Use Agents](https://mlsanity.com/paper/2506.02456/) (2025)
- [BackdoorVLM: A Benchmark for Backdoor Attacks and Defenses on Vision-Language Models](https://mlsanity.com/paper/2511.18921/) (2025)
- [Evolving Contextual Safety in Multi-Modal Large Language Models via Inference-Time Self-Reflective Memory](https://mlsanity.com/paper/2603.15800/) (2026)
- [Simulated Ensemble Attack: Transferring Jailbreaks Across Fine-tuned Vision-Language Models](https://mlsanity.com/paper/2508.01741/) (2025)
- [JPRO: Automated Multimodal Jailbreaking via Multi-Agent Collaboration Framework](https://mlsanity.com/paper/2511.07315/) (2025)
- [Toward Universal and Transferable Jailbreak Attacks on Vision-Language Models](https://mlsanity.com/paper/2602.01025/) (2026)
- [SAVeS: Steering Safety Judgments in Vision-Language Models via Semantic Cues](https://mlsanity.com/paper/2603.19092/) (2026)
- [Visual Exclusivity Attacks: Automatic Multimodal Red Teaming via Agentic Planning](https://mlsanity.com/paper/2603.20198/) (2026)
- [Adversarial Attacks on Multimodal Large Language Models: A Comprehensive Survey](https://mlsanity.com/paper/2603.27918/) (2026)
- [WARD: Adversarially Robust Defense of Web Agents Against Prompt Injections](https://mlsanity.com/paper/2605.15030/) (2026)
- [Multi-turn Jailbreaking Attack in Multi-Modal Large Language Models](https://mlsanity.com/paper/2601.05339/) (2026)
- [PRISM: Robust VLM Alignment with Principled Reasoning for Integrated Safety in Multimodality](https://mlsanity.com/paper/2508.18649/) (2025)
- [Every Picture Tells a Dangerous Story: Memory-Augmented Multi-Agent Jailbreak Attacks on VLMs](https://mlsanity.com/paper/2604.12616/) (2026)
- [LLaVAShield: Safeguarding Multimodal Multi-Turn Dialogues in Vision-Language Models](https://mlsanity.com/paper/2509.25896/) (2025)
- [Understanding Adversarial Transfer: Why Representation-Space Attacks Fail Where Data-Space Attacks Succeed](https://mlsanity.com/paper/2510.01494/) (2025)
- [VERA-V: Variational Inference Framework for Jailbreaking Vision-Language Models](https://mlsanity.com/paper/2510.17759/) (2025)
- [Stealth Fine-Tuning: Efficiently Breaking Alignment in RVLMs Using Self-Generated CoT](https://mlsanity.com/paper/2511.14106/) (2025)
- [Are GUI Agents Focused Enough? Automated Distraction via Semantic-level UI Element Injection](https://mlsanity.com/paper/2604.07831/) (2026)
- [Adversarial Diffusion Across Modalities: A Fusion Survey of Attacks, Defenses, and Evaluation for Text, Vision, and Vision-Language Models](https://mlsanity.com/paper/2606.26566/) (2026)
- [Adversarial Style Optimization: Enhancing VLM Jailbreaks by GRPO-based Stylistic Triggers Optimization](https://mlsanity.com/paper/2607.21619/) (2026)
- [RedDiffuser: Auditing Multimodal Safety Failures in Vision-Language Models via Reinforced Diffusion](https://mlsanity.com/paper/2503.06223/) (2025)
- [Defense-to-Attack: Bypassing Weak Defenses Enables Stronger Jailbreaks in Vision-Language Models](https://mlsanity.com/paper/2509.12724/) (2025)
- [Dictionary-Aligned Concept Control for Safeguarding Multimodal LLMs](https://mlsanity.com/paper/2604.08846/) (2026)
- [ProjLens: Unveiling the Role of Projectors in Multimodal Model Safety](https://mlsanity.com/paper/2604.19083/) (2026)
- [STARE: Step-wise Temporal Alignment and Red-teaming Engine for Multi-modal Toxicity Attack](https://mlsanity.com/paper/2605.00699/) (2026)
- [RedEdit: Agentic Red-Teaming of Image Safety Classifiers via MCTS-Guided Photo-Editing](https://mlsanity.com/paper/2606.06140/) (2026)
- [PHANTOM: A Large-Scale Dataset of Multimodal Adversarial Attacks for Vision-Language Models](https://mlsanity.com/paper/2606.24388/) (2026)
- [V-DEAL: Diagnosing Video Safety De-Calibration as an Understanding-Refusal Coupling Failure](https://mlsanity.com/paper/2607.21151/) (2026)
- [A Multimodal Automatic Redteaming Evaluation based on Atomic Jailbreak Strategy Decoupling and Combination](https://mlsanity.com/paper/2608.04034/) (2026)

## Abstract (arXiv)

As large Vision-Language Models (VLMs) gain prominence, ensuring their safe deployment has become critical. Recent studies have explored VLM robustness against jailbreak attacks-techniques that exploit model vulnerabilities to elicit harmful outputs. However, the limited availability of diverse multimodal data has constrained current approaches to rely heavily on adversarial or manually crafted images derived from harmful text datasets, which often lack effectiveness and diversity across different contexts. In this paper, we propose IDEATOR, a novel jailbreak method that autonomously generates malicious image-text pairs for black-box jailbreak attacks. IDEATOR is grounded in the insight that VLMs themselves could serve as powerful red team models for generating multimodal jailbreak prompts. Specifically, IDEATOR leverages a VLM to create targeted jailbreak texts and pairs them with jailbreak images generated by a state-of-the-art diffusion model. Extensive experiments demonstrate IDEATOR's high effectiveness and transferability, achieving a 94% attack success rate (ASR) in jailbreaking MiniGPT-4 with an average of only 5.34 queries, and high ASRs of 82%, 88%, and 75% when transferred to LLaVA, InstructBLIP, and Chameleon, respectively. Building on IDEATOR's strong transferability and automated process, we introduce the VLJailbreakBench, a safety benchmark comprising 3,654 multimodal jailbreak samples. Our benchmark results on 11 recently released VLMs reveal significant gaps in safety alignment. For instance, our challenge set achieves ASRs of 46.31% on GPT-4o and 19.65% on Claude-3.5-Sonnet, underscoring the urgent need for stronger defenses. VLJailbreakBench is publicly available at https://roywang021.github.io/VLJailbreakBench.