# Image-based Prompt Injection: Hijacking Multimodal LLMs through Visually Embedded Adversarial Instructions

arXiv: https://arxiv.org/abs/2603.03637 · HTML: https://mlsanity.com/paper/2603.03637/

## Key facts

- Cited by 8 papers in the map (data through 14 Oct 2026).
- Placed in MLLM jailbreaks and adversarial robustness, one of 16 directions in Multimodal LLMs / vision-language models.
- Builds on 21 of its 28 references that are in the map.
- First posted on arXiv 4 Mar 2026.

## Placed in

- Multimodal LLMs / vision-language models / [MLLM jailbreaks and adversarial robustness](https://mlsanity.com/mllm/mllm-jailbreaks-and-adversarial-robustness/)
- LLM safety and security / [Vision-language and text-to-image safety](https://mlsanity.com/safety/vision-language-and-text-to-image-safety/)

## Builds on (most cited first, 21 of 21)

- [Not what you've signed up for: Compromising Real-World LLM-Integrated Applications with Indirect Prompt Injection](https://mlsanity.com/paper/2302.12173/) (2023)
- [Vision-Language Models for Vision Tasks: A Survey](https://mlsanity.com/paper/2304.00685/) (2023)
- [Ignore Previous Prompt: Attack Techniques For Language Models](https://mlsanity.com/paper/2211.09527/) (2022)
- [FigStep: Jailbreaking Large Vision-Language Models via Typographic Visual Prompts](https://mlsanity.com/paper/2311.05608/) (2023)
- [Visual Adversarial Examples Jailbreak Aligned Large Language Models](https://mlsanity.com/paper/2306.13213/) (2023)
- [Are aligned neural networks adversarially aligned?](https://mlsanity.com/paper/2306.15447/) (2023)
- [On Evaluating Adversarial Robustness of Large Vision-Language Models](https://mlsanity.com/paper/2305.16934/) (2023)
- [MM-SafetyBench: A Benchmark for Safety Evaluation of Multimodal Large Language Models](https://mlsanity.com/paper/2311.17600/) (2023)
- [Jailbreak in pieces: Compositional Adversarial Attacks on Multi-Modal Language Models](https://mlsanity.com/paper/2307.14539/) (2023)
- [Security and Privacy Challenges of Large Language Models: A Survey](https://mlsanity.com/paper/2402.00888/) (2024)
- [How Robust is Google's Bard to Adversarial Image Attacks?](https://mlsanity.com/paper/2309.11751/) (2023)
- [Image Hijacks: Adversarial Images can Control Generative Models at Runtime](https://mlsanity.com/paper/2309.00236/) (2023)
- [Agent Smith: A Single Image Can Jailbreak One Million Multimodal LLM Agents Exponentially Fast](https://mlsanity.com/paper/2402.08567/) (2024)
- [Jailbreaking Attack against Multimodal Large Language Model](https://mlsanity.com/paper/2402.02309/) (2024)
- [PaLI-3 Vision Language Models: Smaller, Faster, Stronger](https://mlsanity.com/paper/2310.09199/) (2023)
- [Abusing Images and Sounds for Indirect Instruction Injection in Multi-Modal LLMs](https://mlsanity.com/paper/2307.10490/) (2023)
- [Eyes Closed, Safety On: Protecting Multimodal LLMs via Image-to-Text Transformation](https://mlsanity.com/paper/2403.09572/) (2024)
- [An Early Categorization of Prompt Injection Attacks on Large Language Models](https://mlsanity.com/paper/2402.00898/) (2024)
- [Safeguarding Vision-Language Models Against Patched Visual Prompt Injectors](https://mlsanity.com/paper/2405.10529/) (2024)
- [Empirical Analysis of Large Vision-Language Models against Goal Hijacking via Visual Prompt Injection](https://mlsanity.com/paper/2408.03554/) (2024)
- [Con Instruction: Universal Jailbreaking of Multimodal Large Language Models via Non-Textual Modalities](https://mlsanity.com/paper/2506.00548/) (2025)

## Built on by (most cited first, 8 of 8)

- [From Prompt to Physical Actuation: Holistic Threat Modeling of LLM-Enabled Robotic Systems](https://mlsanity.com/paper/2604.27267/) (2026)
- [Hallucination as Exploit: Evidence-Carrying Multimodal Agents](https://mlsanity.com/paper/2605.19192/) (2026)
- [Hierarchical Attacks for Multi-Modal Multi-Agent Reasoning](https://mlsanity.com/paper/2605.13213/) (2026)
- [When Prompts Control Robots: Prompt Injection Attacks in Multi-Agent Robotic Systems](https://mlsanity.com/paper/2608.00747/) (2026)
- [Where Do LLM-based Systems Break? A System-Level Security Framework for Risk Assessment and Treatment](https://mlsanity.com/paper/2603.07460/) (2026)
- [Repeat-After-Me: Black-Box Adaptive Visual Prompt Injection](https://mlsanity.com/paper/2609.04533/) (2026)
- [MM-IFEval-Pro: A Multilingual and Attack-Resistant Benchmark for Instruction-Following in Vision-Language Models](https://mlsanity.com/paper/2609.04859/) (2026)
- [Render Before Reading: Visual Rendering as a Prompt Injection Defense](https://mlsanity.com/paper/2609.36121/) (2026)

## Abstract (arXiv)

Multimodal Large Language Models (MLLMs) integrate vision and text to power applications, but this integration introduces new vulnerabilities. We study Image-based Prompt Injection (IPI), a black-box attack in which adversarial instructions are embedded into natural images to override model behavior. Our end-to-end IPI pipeline incorporates segmentation-based region selection, adaptive font scaling, and background-aware rendering to conceal prompts from human perception while preserving model interpretability. Using the COCO dataset and GPT-4-turbo, we evaluate 12 adversarial prompt strategies and multiple embedding configurations. The results show that IPI can reliably manipulate the output of the model, with the most effective configuration achieving up to 64% attack success under stealth constraints. These findings highlight IPI as a practical threat in black-box settings and underscore the need for defenses against multimodal prompt injection.