mlsanityData to 7 Oct 2026

Image-based Prompt Injection: Hijacking Multimodal LLMs through Visually Embedded Adversarial Instructions

arXiv 2603.03637 ↗First posted 4 Mar 2026cs.CV cs.AI cs.CRcited by 8

On the map: Multimodal LLMs / vision-language models → MLLM jailbreaks and adversarial robustness

also in 1 more
Abstract (arXiv)

Multimodal Large Language Models (MLLMs) integrate vision and text to power applications, but this integration introduces new vulnerabilities. We study Image-based Prompt Injection (IPI), a black-box attack in which adversarial instructions are embedded into natural images to override model behavior. Our end-to-end IPI pipeline incorporates segmentation-based region selection, adaptive font scaling, and background-aware rendering to conceal prompts from human perception while preserving model interpretability. Using the COCO dataset and GPT-4-turbo, we evaluate 12 adversarial prompt strategies and multiple embedding configurations. The results show that IPI can reliably manipulate the output of the model, with the most effective configuration achieving up to 64% attack success under stealth constraints. These findings highlight IPI as a practical threat in black-box settings and underscore the need for defenses against multimodal prompt injection.

Key facts. Cited by 8 papers in the map (data through 7 Oct 2026). Placed in MLLM jailbreaks and adversarial robustness, one of 16 directions in Multimodal LLMs / vision-language models. Builds on 21 of its 28 references that are in the map. First posted on arXiv 4 Mar 2026.

Builds on

21 of 28 references.

Closest ancestors · 12

Same direction, newest first

  1. 2025Con Instruction: Universal Jailbreaking of Multimodal Large Language Models via Non-Textual Modalities31 May
  2. 2024Empirical Analysis of Large Vision-Language Models against Goal Hijacking via Visual Prompt Injection7 Aug
  3. 2024Safeguarding Vision-Language Models Against Patched Visual Prompt Injectors17 May
  4. 2024Eyes Closed, Safety On: Protecting Multimodal LLMs via Image-to-Text Transformation14 Mar
  5. 2024Agent Smith: A Single Image Can Jailbreak One Million Multimodal LLM Agents Exponentially Fast13 Feb
  6. 2023MM-SafetyBench: A Benchmark for Safety Evaluation of Multimodal Large Language Models29 Nov
  7. 2023FigStep: Jailbreaking Large Vision-Language Models via Typographic Visual Prompts9 Nov
  8. 2023How Robust is Google's Bard to Adversarial Image Attacks?21 Sep
  9. 2023Jailbreak in pieces: Compositional Adversarial Attacks on Multi-Modal Language Models26 Jul
  10. 2023Abusing Images and Sounds for Indirect Instruction Injection in Multi-Modal LLMs19 Jul
  11. 2023Visual Adversarial Examples Jailbreak Aligned Large Language Models22 Jun
  12. 2023On Evaluating Adversarial Robustness of Large Vision-Language Models26 May

Other references · 9

  1. 2023Not what you've signed up for: Compromising Real-World LLM-Integrated Applications with Indirect Prompt Injectioncited by 1,339
  2. 2023Vision-Language Models for Vision Tasks: A Surveycited by 834
  3. 2022Ignore Previous Prompt: Attack Techniques For Language Modelscited by 760
  4. 2023Are aligned neural networks adversarially aligned?cited by 305
  5. 2024Security and Privacy Challenges of Large Language Models: A Surveycited by 179
  6. 2023Image Hijacks: Adversarial Images can Control Generative Models at Runtimecited by 165
  7. 2024Jailbreaking Attack against Multimodal Large Language Modelcited by 128
  8. 2023PaLI-3 Vision Language Models: Smaller, Faster, Strongercited by 125
  9. 2024An Early Categorization of Prompt Injection Attacks on Large Language Modelscited by 28

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2026: 8.

8 papers.

  1. 2026From Prompt to Physical Actuation: Holistic Threat Modeling of LLM-Enabled Robotic Systemscited by 3
  2. 2026Hallucination as Exploit: Evidence-Carrying Multimodal Agentscited by 2
  3. 2026Hierarchical Attacks for Multi-Modal Multi-Agent Reasoningcited by 1
  4. 2026When Prompts Control Robots: Prompt Injection Attacks in Multi-Agent Robotic Systemscited by 1
  5. 2026Where Do LLM-based Systems Break? A System-Level Security Framework for Risk Assessment and Treatmentcited by 0
  6. 2026Repeat-After-Me: Black-Box Adaptive Visual Prompt Injectioncited by 0
  7. 2026MM-IFEval-Pro: A Multilingual and Attack-Resistant Benchmark for Instruction-Following in Vision-Language Modelscited by 0
  8. 2026Render Before Reading: Visual Rendering as a Prompt Injection Defensecited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2026Covert Visual Prompt Injection against Commercial Multimodal Large Language Models
  2. 2026A Cross-Modal Prompt Injection Attack against Large Vision-Language Models with Image-Only Perturbation
  3. 2025Manipulating Multimodal Agents via Cross-Modal Prompt Injection
  4. 2025An Image Is Worth Ten Thousand Words: Verbose-Text Induction Attacks on VLMs
  5. 2026Adversarial Attacks on Multimodal Large Language Models: A Comprehensive Survey
  6. 2024Unbridled Icarus: A Survey of the Potential Perils of Image Inputs in Multimodal Large Language Model Security
  7. 2025ARGUS: Defending Against Multimodal Indirect Prompt Injection via Steering Instruction-Following Behavior
  8. 2025Beyond Text: Multimodal Jailbreaking of Vision-Language and Audio Models through Perceptually Simple Transformations