mlsanityData to 7 Oct 2026

Arondight: Red Teaming Large Vision Language Models with Auto-generated Multi-modal Jailbreak Prompts

arXiv 2407.15050 ↗First posted 21 Jul 2024cs.LG cs.AI cs.CRcited by 39

On the map: Multimodal LLMs / vision-language models → MLLM jailbreaks and adversarial robustness

also in 1 more
Abstract (arXiv)

Large Vision Language Models (VLMs) extend and enhance the perceptual abilities of Large Language Models (LLMs). Despite offering new possibilities for LLM applications, these advancements raise significant security and ethical concerns, particularly regarding the generation of harmful content. While LLMs have undergone extensive security evaluations with the aid of red teaming frameworks, VLMs currently lack a well-developed one. To fill this gap, we introduce Arondight, a standardized red team framework tailored specifically for VLMs. Arondight is dedicated to resolving issues related to the absence of visual modality and inadequate diversity encountered when transitioning existing red teaming methodologies from LLMs to VLMs. Our framework features an automated multi-modal jailbreak attack, wherein visual jailbreak prompts are produced by a red team VLM, and textual prompts are generated by a red team LLM guided by a reinforcement learning agent. To enhance the comprehensiveness of VLM security evaluation, we integrate entropy bonuses and novelty reward metrics. These elements incentivize the RL agent to guide the red team LLM in creating a wider array of diverse and previously unseen test cases. Our evaluation of ten cutting-edge VLMs exposes significant security vulnerabilities, particularly in generating toxic images and aligning multi-modal prompts. In particular, our Arondight achieves an average attack success rate of 84.5% on GPT-4 in all fourteen prohibited scenarios defined by OpenAI in terms of generating toxic text. For a clearer comparison, we also categorize existing VLMs based on their safety levels and provide corresponding reinforcement recommendations. Our multimodal prompt dataset and red team code will be released after ethics committee approval. CONTENT WARNING: THIS PAPER CONTAINS HARMFUL MODEL RESPONSES.

Key facts. Cited by 39 papers in the map (data through 7 Oct 2026). Placed in MLLM jailbreaks and adversarial robustness, one of 16 directions in Multimodal LLMs / vision-language models. Builds on 31 of its 51 references that are in the map. First posted on arXiv 21 Jul 2024.

Builds on

31 of 51 references.

Closest ancestors · 1

Same direction, newest first

  1. 2022Towards Adversarial Attack on Vision-Language Pre-training Models19 Jun

Other references · 30

  1. 2019Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformercited by 14,491
  2. 2019BERTScore: Evaluating Text Generation with BERTcited by 5,462
  3. 2022BLIP: Bootstrapping Language-Image Pre-training for Unified Vision-Language Understanding and Generationcited by 4,310
  4. 2023Universal and Transferable Adversarial Attacks on Aligned Language Modelscited by 2,914
  5. 2023Jailbroken: How Does LLM Safety Training Fail?cited by 1,565
  6. 2023Jailbreaking Black Box Large Language Models in Twenty Queriescited by 1,326
  7. 2021GLM: General Language Model Pretraining with Autoregressive Blank Infillingcited by 1,026
  8. 2022Red Teaming Language Models with Language Modelscited by 983
  9. 2021Multimodal Few-Shot Learning with Frozen Language Modelscited by 657
  10. 2023LLaMA-Adapter V2: Parameter-Efficient Visual Instruction Modelcited by 602
  11. 2023Prompt Injection attack against LLM-integrated Applicationscited by 568
  12. 2017Adversarial Examples Are Not Easily Detected: Bypassing Ten Detection Methodscited by 542
18 more
  1. 2023GPTFUZZER: Red Teaming Large Language Models with Auto-Generated Jailbreak Promptscited by 520
  2. 2023Jailbreaking ChatGPT via Prompt Engineering: An Empirical Studycited by 478
  3. 2023Jailbreak and Guard Aligned Language Models with Only Few In-Context Demonstrationscited by 401
  4. 2023DeepInception: Hypnotize Large Language Model to Be Jailbreakercited by 337
  5. 2020BAE: BERT-based Adversarial Examples for Text Classificationcited by 316
  6. 2023Red-Teaming Large Language Models using Chain of Utterances for Safety-Alignmentcited by 213
  7. 2023Survey of Vulnerabilities in Large Language Models Revealed by Adversarial Attackscited by 189
  8. 2023SneakyPrompt: Jailbreaking Text-to-image Generative Modelscited by 175
  9. 2021On the Adversarial Robustness of Vision Transformerscited by 104
  10. 2021Spinning Language Models: Risks of Propaganda-As-A-Service and Countermeasurescited by 69
  11. 2023Red Teaming Language Model Detectors with Language Modelscited by 55
  12. 2022Fine-mixing: Mitigating Backdoors in Fine-tuned Language Modelscited by 50
  13. 2023You Only Prompt Once: On the Capabilities of Prompt Learning on Large Language Models to Tackle Toxic Contentcited by 37
  14. 2022Reducing the Vision and Language Bias for Temporal Sentence Groundingcited by 35
  15. 2023Training-free Lexical Backdoor Attacks on Language Modelscited by 27
  16. 2022IDEA: Increasing Text Diversity via Online Multi-Label Recognition for Vision-Language Pre-trainingcited by 14
  17. 2022MVPTR: Multi-Level Semantic Alignment for Vision-Language Pre-Training via Multi-Stage Learningcited by 13
  18. 2023Food-500 Cap: A Fine-Grained Food Caption Benchmark for Evaluating Vision-Language Modelscited by 10

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2024: 7; 2025: 24; 2026: 8.

39 papers.

Sort:

By area:

  1. 2025LLM Post-Training: A Deep Dive into Reasoning Large Language Modelscited by 129
  2. 2024SG-Bench: Evaluating LLM Safety Generalization Across Diverse Tasks and Prompt Typescited by 58
  3. 2024Visual Prompting in Multimodal Large Language Models: A Surveycited by 52
  4. 2024IDEATOR: Jailbreaking and Benchmarking Large Vision-Language Models Using Themselvescited by 34
  5. 2024Jailbreak Attacks and Defenses against Multimodal Generative Models: A Surveycited by 31
  6. 2025Distraction is All You Need for Multimodal Large Language Model Jailbreakingcited by 31
  7. 2024PBI-Attack: Prior-Guided Bimodal Interactive Black-Box Jailbreak Attack for Toxicity Maximizationcited by 17
  8. 2025Are Vision-Language Models Safe in the Wild? A Meme-Based Benchmark Studycited by 12
  9. 2025JPS: Jailbreak Multimodal Large Language Models with Collaborative Visual Perturbation and Textual Steeringcited by 10
  10. 2024Safe + Safe = Unsafe? Exploring How Safe Images Can Be Exploited to Jailbreak Large Vision-Language Modelscited by 9
  11. 2024Recent advancements in LLM Red-Teaming: Techniques, Defenses, and Ethical Considerationscited by 8
  12. 2025Speech-Audio Compositional Attacks on Multimodal LLMs and Their Mitigation with SALMONN-Guardcited by 8
27 more
  1. 2025Implicit Jailbreak Attacks via Cross-Modal Information Concealment on Vision-Language Modelscited by 5
  2. 2025Mind the Gesture: Evaluating AI Sensitivity to Culturally Offensive Non-Verbal Gesturescited by 4
  3. 2025REVEAL: Multi-turn Evaluation of Image-Input Harms for Vision LLMcited by 4
  4. 2025GenBreak: Red Teaming Text-to-Image Generators Using Large Language Modelscited by 4
  5. 2025SoK: Systematizing LLM Prompt Security: Taxonomies, Datasets, and Unified Evaluation of Attacks and Defensescited by 4
  6. 2025CrossGuard: Safeguarding MLLMs against Joint-Modal Implicit Malicious Attackscited by 4
  7. 2026Tone Matters: The Impact of Linguistic Tone on Hallucination in VLMscited by 4
  8. 2025Bootstrapping LLM Robustness for VLM Safety via Reducing the Pretraining Modality Gapcited by 3
  9. 2025Toxicity in Online Platforms and AI Systems: A Survey of Needs, Challenges, Mitigations, and Future Directionscited by 3
  10. 2025JPRO: Automated Multimodal Jailbreaking via Multi-Agent Collaboration Frameworkcited by 3
  11. 2026Adversarial Attacks on Multimodal Large Language Models: A Comprehensive Surveycited by 3
  12. 2026LLM-as-Judge Framework for Evaluating Tone-Induced Hallucination in Vision-Language Modelscited by 3
  13. 2025Phonetic Perturbations Reveal Tokenizer-Rooted Safety Gaps in LLMscited by 2
  14. 2025Watch, Listen, Understand, Mislead: Tri-modal Adversarial Attacks on Short Videos for Content Appropriateness Evaluationcited by 2
  15. 2025Resource Consumption Red-Teaming for Large Vision-Language Modelscited by 2
  16. 2025JaiLIP: Jailbreaking Vision-Language Models via Loss Guided Image Perturbationcited by 2
  17. 2025Medusa: Cross-Modal Transferable Adversarial Attacks on Multimodal Medical Retrieval-Augmented Generationcited by 2
  18. 2026TreeTeaming: Autonomous Red-Teaming of Vision-Language Models via Hierarchical Strategy Explorationcited by 2
  19. 2025Benign-to-Toxic Jailbreaking: Inducing Harmful Responses from Harmless Promptscited by 1
  20. 2025VERA-V: Variational Inference Framework for Jailbreaking Vision-Language Modelscited by 1
  21. 2026Robustness of Vision Language Models Against Split-Image Harmful Input Attackscited by 1
  22. 2025TestDG: Test-time Domain Generalization for Continual Test-time Adaptationcited by 0
  23. 2025RedTopic: Toward Topic-Diverse Red Teaming of Large Language Modelscited by 0
  24. 2025Pruning Cannot Hurt Robustness: Certified Trade-offs in Reinforcement Learningcited by 0
  25. 2026Single-Sample Black-Box Membership Inference Attack against Vision-Language Models via Cross-modal Semantic Alignmentcited by 0
  26. 2026Overloading Large Vision-Language Models for Jailbreakingcited by 0
  27. 2026Fully Unleashing the Multimodal Attacker: Meta-Adaptive Jailbreaking of Vision-Language Modelscited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2026REALM: A Unified Red-Teaming Benchmark for Physical-World VLMs
  2. 2024Red Teaming Visual Language Models
  3. 2025ARMs: Adaptive Red-Teaming Agent against Multimodal Models with Plug-and-Play Attacks
  4. 2025When Data Manipulation Meets Attack Goals: An In-depth Survey of Attacks for VLMs
  5. 2024ImgTrojan: Jailbreaking Vision-Language Models with ONE Image
  6. 2025A Survey of Safety on Large Vision-Language Models: Attacks, Defenses and Evaluations
  7. 2024JailbreakZoo: Survey, Landscapes, and Horizons in Jailbreaking Large Language and Vision-Language Models
  8. 2024A Survey of Attacks on Large Vision-Language Models: Resources, Advances, and Future Trends