mlsanityData to 7 Oct 2026

Image Hijacks: Adversarial Images can Control Generative Models at Runtime

arXiv 2309.00236 ↗First posted 1 Sep 2023cs.LG cs.CL cs.CRcited by 165

Not on the map. Why?

Cited by 97 in Multimodal LLMs / vision-language models, but placed by its own abstract, which didn’t fit.

Abstract (arXiv)

Are foundation models secure against malicious actors? In this work, we focus on the image input to a vision-language model (VLM). We discover image hijacks, adversarial images that control the behaviour of VLMs at inference time, and introduce the general Behaviour Matching algorithm for training image hijacks. From this, we derive the Prompt Matching method, allowing us to train hijacks matching the behaviour of an arbitrary user-defined text prompt (e.g. 'the Eiffel Tower is now located in Rome') using a generic, off-the-shelf dataset unrelated to our choice of prompt. We use Behaviour Matching to craft hijacks for four types of attack, forcing VLMs to generate outputs of the adversary's choice, leak information from their context window, override their safety training, and believe false statements. We study these attacks against LLaVA, a state-of-the-art VLM based on CLIP and LLaMA-2, and find that all attack types achieve a success rate of over 80%. Moreover, our attacks are automated and require only small image perturbations.

Key facts. Cited by 165 papers in the map (data through 7 Oct 2026). Builds on 34 of its 53 references that are in the map. First posted on arXiv 1 Sep 2023.

Builds on

34 of 53 references.

References · 34

  1. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,456
  2. 2023Llama 2: Open Foundation and Fine-Tuned Chat Modelscited by 12,865
  3. 2015Distilling the Knowledge in a Neural Networkcited by 10,076
  4. 2023Visual Instruction Tuningcited by 7,792
  5. 2014Explaining and Harnessing Adversarial Examplescited by 7,287
  6. 2023BLIP-2: Bootstrapping Language-Image Pre-training with Frozen Image Encoders and Large Language Modelscited by 6,069
  7. 2013Intriguing properties of neural networkscited by 5,114
  8. 2021The Power of Scale for Parameter-Efficient Prompt Tuningcited by 3,769
  9. 2023Universal and Transferable Adversarial Attacks on Aligned Language Modelscited by 2,914
  10. 2023InstructBLIP: Towards General-purpose Vision-Language Models with Instruction Tuningcited by 2,855
  11. 2023Jailbroken: How Does LLM Safety Training Fail?cited by 1,565
  12. 2023Not what you've signed up for: Compromising Real-World LLM-Integrated Applications with Indirect Prompt Injectioncited by 1,339
22 more
  1. 2019Certified Adversarial Robustness via Randomized Smoothingcited by 1,324
  2. 2018Obfuscated Gradients Give a False Sense of Security: Circumventing Defenses to Adversarial Examplescited by 1,259
  3. 2014Deep Neural Networks are Easily Fooled: High Confidence Predictions for Unrecognizable Imagescited by 774
  4. 2022Ignore Previous Prompt: Attack Techniques For Language Modelscited by 760
  5. 2020RobustBench: a standardized adversarial robustness benchmarkcited by 659
  6. 2023Prompt Injection attack against LLM-integrated Applicationscited by 568
  7. 2022Mind the Gap: Understanding the Modality Gap in Multi-modal Contrastive Representation Learningcited by 553
  8. 2017Countering Adversarial Images using Input Transformationscited by 517
  9. 2017Adversarial Patchcited by 514
  10. 2020BERT-ATTACK: Adversarial Attack Against BERT Using BERTcited by 488
  11. 2023Augmented Language Models: a Surveycited by 398
  12. 2023Are aligned neural networks adversarially aligned?cited by 305
  13. 2023On Evaluating Adversarial Robustness of Large Vision-Language Modelscited by 281
  14. 2020Uncovering the Limits of Adversarial Training against Norm-Bounded Adversarial Examplescited by 275
  15. 2023Jailbreak in pieces: Compositional Adversarial Attacks on Multi-Modal Language Modelscited by 260
  16. 2023Better Diffusion Models Further Improve Adversarial Trainingcited by 207
  17. 2023Automatically Auditing Large Language Models via Discrete Optimizationcited by 187
  18. 2023On the Adversarial Robustness of Multi-Modal Foundation Modelscited by 142
  19. 2022(Certified!!) Adversarial Robustness for Free!cited by 130
  20. 2023Abusing Images and Sounds for Indirect Instruction Injection in Multi-Modal LLMscited by 117
  21. 2023From Prompt Injections to SQL Injection Attacks: How Protected is Your LLM-Integrated Web Application?cited by 53
  22. 2021Random Noise Defense Against Query-Based Black-Box Attackscited by 46

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2023: 7; 2024: 51; 2025: 58; 2026: 49.

165 papers.

Sort:

By area:

  1. 2024HarmBench: A Standardized Evaluation Framework for Automated Red Teaming and Robust Refusalcited by 1,330
  2. 2023FigStep: Jailbreaking Large Vision-Language Models via Typographic Visual Promptscited by 368
  3. 2024A StrongREJECT for Empty Jailbreakscited by 352
  4. 2024Improving Alignment and Robustness with Circuit Breakerscited by 302
  5. 2023Survey of Vulnerabilities in Large Language Models Revealed by Adversarial Attackscited by 189
  6. 2023How Robust is Google's Bard to Adversarial Image Attacks?cited by 177
  7. 2024Agent Smith: A Single Image Can Jailbreak One Million Multimodal LLM Agents Exponentially Fastcited by 149
  8. 2024Jailbreaking Attack against Multimodal Large Language Modelcited by 128
  9. 2025Advances and Challenges in Foundation Agents: From Brain-Inspired Intelligence to Evolutionary, Collaborative, and Safe Systemscited by 125
  10. 2024Robust CLIP: Unsupervised Adversarial Fine-Tuning of Vision Embeddings for Robust Large Vision-Language Modelscited by 119
  11. 2024Dissecting Adversarial Robustness of Multimodal LM Agentscited by 118
  12. 2024Eyes Closed, Safety On: Protecting Multimodal LLMs via Image-to-Text Transformationcited by 114
38 more
  1. 2024Coercing LLMs to do and reveal (almost) anythingcited by 88
  2. 2024Defending Against Unforeseen Failure Modes with Latent Adversarial Trainingcited by 82
  3. 2024SPA-VL: A Comprehensive Safety Preference Alignment Dataset for Vision Language Modelcited by 82
  4. 2024A Survey of Attacks on Large Vision-Language Models: Resources, Advances, and Future Trendscited by 82
  5. 2024Red-Teaming for Generative AI: Silver Bullet or Security Theater?cited by 81
  6. 2024PsySafe: A Comprehensive Framework for Psychological-based Attack, Defense, and Evaluation of Multi-agent System Safetycited by 79
  7. 2024Visibility into AI Agentscited by 77
  8. 2024Best-of-N Jailbreakingcited by 71
  9. 2024Safety of Multimodal Large Language Models on Images and Textscited by 70
  10. 2024JailbreakZoo: Survey, Landscapes, and Horizons in Jailbreaking Large Language and Vision-Language Modelscited by 67
  11. 2024Exploring the Frontier of Vision-Language Models: A Survey of Current Methodologies and Future Directionscited by 66
  12. 2025Safety at Scale: A Comprehensive Survey of Large Model and Agent Safetycited by 63
  13. 2023Jailbreaking GPT-4V via Self-Adversarial Attacks with System Promptscited by 62
  14. 2024White-box Multimodal Jailbreaks Against Large Vision-Language Modelscited by 57
  15. 2024ETA: Evaluating Then Aligning Safety of Vision Language Models at Inference Timecited by 57
  16. 2024Query-Based Adversarial Prompt Generationcited by 55
  17. 2024Shadowcast: Stealthy Data Poisoning Attacks Against Vision-Language Modelscited by 48
  18. 2023Can Language Models be Instructed to Protect Personal Information?cited by 46
  19. 2023Can LLMs Follow Simple Rules?cited by 45
  20. 2025Jailbreaking Multimodal Large Language Models via Shuffle Inconsistencycited by 42
  21. 2025Adversarial Attacks against Closed-Source MLLMs via Feature Optimal Alignmentcited by 42
  22. 2024Adversarial Robustness Limits via Scaling-Law and Human-Alignment Studiescited by 38
  23. 2024Chain of Attack: On the Robustness of Vision-Language Models Against Transfer-Based Adversarial Attackscited by 37
  24. 2024Test-Time Backdoor Attacks on Multimodal Large Language Modelscited by 35
  25. 2024LLMs Meet Multimodal Generation and Editing: A Surveycited by 35
  26. 2024Visual Adversarial Attack on Vision-Language Models for Autonomous Drivingcited by 35
  27. 2024IDEATOR: Jailbreaking and Benchmarking Large Vision-Language Models Using Themselvescited by 34
  28. 2024Jailbreak Attacks and Defenses against Multimodal Generative Models: A Surveycited by 31
  29. 2025HiddenDetect: Detecting Jailbreak Attacks against Large Vision-Language Models via Monitoring Hidden Statescited by 29
  30. 2024BlueSuffix: Reinforced Blue Teaming for Vision-Language Models Against Jailbreak Attackscited by 28
  31. 2024Failures to Find Transferable Image Jailbreaks Between Vision-Language Modelscited by 27
  32. 2024Stop Reasoning! When Multimodal LLM with Chain-of-Thought Reasoning Meets Adversarial Imagecited by 26
  33. 2024B-AVIBench: Towards Evaluating the Robustness of Large Vision-Language Model on Black-box Adversarial Visual-Instructionscited by 26
  34. 2024Safeguarding Vision-Language Models Against Patched Visual Prompt Injectorscited by 26
  35. 2025Video-SafetyBench: A Benchmark for Safety Evaluation of Video LVLMscited by 25
  36. 2024Benchmarking Large Multimodal Models against Common Corruptionscited by 22
  37. 2024From LLMs to MLLMs: Exploring the Landscape of Multimodal Jailbreakingcited by 22
  38. 2023InstructTA: Instruction-Tuned Targeted Attack for Large Vision-Language Modelscited by 21

50 most cited of 165 All 165 (JSON)

Similar, not cited

Close in wording, no citation either way

  1. 2025Attention! Your Vision Language Model Could Be Maliciously Manipulated
  2. 2024Exploring Visual Vulnerabilities via Multi-Loss Adversarial Search for Jailbreaking Vision-Language Models
  3. 2026Attention Hijacking: Response Manipulation Across Queries in Vision-Language Models
  4. 2024Break the Visual Perception: Adversarial Attacks Targeting Encoded Visual Tokens of Large Vision-Language Models
  5. 2026On Success and Simplicity: A Second Look at Transferable Vision-Language Attack Pipeline
  6. 2025When Data Manipulation Meets Attack Goals: An In-depth Survey of Attacks for VLMs
  7. 2024Empirical Analysis of Large Vision-Language Models against Goal Hijacking via Visual Prompt Injection
  8. 2025BackdoorVLM: A Benchmark for Backdoor Attacks and Defenses on Vision-Language Models