mlsanityData to 7 Oct 2026

Covert Visual Prompt Injection against Commercial Multimodal Large Language Models

arXiv 2603.29418 ↗First posted 31 Mar 2026cs.CV cs.AIcited by 0

On the map: Multimodal LLMs / vision-language models → MLLM jailbreaks and adversarial robustness

also in 1 more
Abstract (arXiv)

Although multimodal large language models (MLLMs) are increasingly deployed in real-world applications, their instruction-following behavior leaves them vulnerable to prompt injection attacks. Existing prompt injection methods predominantly rely on textual prompts or perceptible visual prompts that are observable by human users. In this work, we study imperceptible visual prompt injection against powerful closed-source MLLMs, where adversarial instructions are embedded in the visual modality. Our method adaptively embeds the malicious prompt into the input image via a bounded text overlay to provide semantic guidance. Meanwhile, the imperceptible visual perturbation is iteratively optimized to align the feature representation of the attacked image with those of the malicious visual and textual targets at both coarse- and fine-grained levels. Specifically, the visual target is instantiated as a text-rendered image and progressively refined during optimization to more faithfully represent the desired semantics and improve transferability. Extensive experiments on two multimodal understanding tasks across multiple closed-source MLLMs demonstrate the superior performance of our approach compared to existing methods.

Key facts. No citing papers in the map yet (data through 7 Oct 2026). Placed in MLLM jailbreaks and adversarial robustness, one of 16 directions in Multimodal LLMs / vision-language models. Builds on 43 of its 48 references that are in the map. First posted on arXiv 31 Mar 2026.

Builds on

43 of 48 references.

Closest ancestors · 11

Same direction, newest first

  1. 2025ARGUS: Defending Against Multimodal Indirect Prompt Injection via Steering Instruction-Following Behavior5 Dec
  2. 2025Manipulating Multimodal Agents via Cross-Modal Prompt Injection19 Apr
  3. 2025A Frustratingly Simple Yet Highly Effective Attack Baseline: Over 90% Success Rate Against the Strong Black-box Models of GPT-4.5/4o/o113 Mar
  4. 2024Chain of Attack: On the Robustness of Vision-Language Models Against Transfer-Based Adversarial Attacks24 Nov
  5. 2024Break the Visual Perception: Adversarial Attacks Targeting Encoded Visual Tokens of Large Vision-Language Models9 Oct
  6. 2024AnyAttack: Towards Large-scale Self-supervised Adversarial Attacks on Vision-language Models7 Oct
  7. 2024Empirical Analysis of Large Vision-Language Models against Goal Hijacking via Visual Prompt Injection7 Aug
  8. 2024Safety of Multimodal Large Language Models on Images and Texts1 Feb
  9. 2023On the Robustness of Large Multimodal Models Against Image Adversarial Attacks6 Dec
  10. 2023Visual Adversarial Examples Jailbreak Aligned Large Language Models22 Jun
  11. 2023On Evaluating Adversarial Robustness of Large Vision-Language Models26 May

Other references · 32

  1. 2017Attention Is All You Needcited by 59,932
  2. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,456
  3. 2021High-Resolution Image Synthesis with Latent Diffusion Modelscited by 18,694
  4. 2023GPT-4 Technical Reportcited by 18,132
  5. 2023Visual Instruction Tuningcited by 7,792
  6. 2014Explaining and Harnessing Adversarial Examplescited by 7,287
  7. 2023BLIP-2: Bootstrapping Language-Image Pre-training with Frozen Image Encoders and Large Language Modelscited by 6,069
  8. 2022LAION-5B: An open large-scale dataset for training next generation image-text modelscited by 4,131
  9. 2024Gemini 1.5: Unlocking multimodal understanding across millions of tokens of contextcited by 3,175
  10. 2022Emergent Abilities of Large Language Modelscited by 2,483
  11. 2022Learn to Explain: Multimodal Reasoning via Thought Chains for Science Question Answeringcited by 1,881
  12. 2016Adversarial examples in the physical worldcited by 1,762
20 more
  1. 2019Certified Adversarial Robustness via Randomized Smoothingcited by 1,324
  2. 2023Prompt Injection attack against LLM-integrated Applicationscited by 568
  3. 2023Are Emergent Abilities of Large Language Models a Mirage?cited by 515
  4. 2022Emergent Analogical Reasoning in Large Language Modelscited by 239
  5. 2023Benchmarking and Defending Against Indirect Prompt Injection Attacks on Large Language Modelscited by 237
  6. 2017Formal Guarantees on the Robustness of a Classifier against Adversarial Manipulationcited by 182
  7. 2024Agent S: An Open Agentic Framework that Uses Computers Like a Humancited by 182
  8. 2023Image Hijacks: Adversarial Images can Control Generative Models at Runtimecited by 165
  9. 2023ManipLLM: Embodied Multimodal Large Language Model for Object-Centric Robotic Manipulationcited by 157
  10. 2024Optimization-based Prompt Injection Attack to LLM-as-a-Judgecited by 123
  11. 2024Dissecting Adversarial Robustness of Multimodal LM Agentscited by 118
  12. 2024Natural Language Understanding and Inference with MLLM in Visual Question Answering: A Surveycited by 74
  13. 2023Evaluating the Instruction-Following Robustness of Large Language Models to Prompt Injectioncited by 53
  14. 2025PromptShield: Deployable Detection for Prompt Injection Attackscited by 43
  15. 2025Adversarial Attacks against Closed-Source MLLMs via Feature Optimal Alignmentcited by 42
  16. 2024Personalizing Multimodal Large Language Models for Image Captioning: An Experimental Analysiscited by 23
  17. 2024Mitigating the Curse of Dimensionality for Certified Robustness via Dual Randomized Smoothingcited by 15
  18. 2025UniGuardian: A Unified Defense for Detecting Prompt Injection, Backdoor Attacks and Adversarial Attacks in Large Language Modelscited by 12
  19. 2025Exploring Typographic Visual Prompts Injection Threats in Cross-Modality Generation Modelscited by 12
  20. 2025Invisible Injections: Exploiting Vision-Language Models Through Steganographic Prompt Embeddingcited by 5

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2026Image-based Prompt Injection: Hijacking Multimodal LLMs through Visually Embedded Adversarial Instructions
  2. 2026A Cross-Modal Prompt Injection Attack against Large Vision-Language Models with Image-Only Perturbation
  3. 2026Leave My Images Alone: Preventing Multi-Modal Large Language Models from Analyzing Images via Visual Prompt Injection
  4. 2025Beyond Text: Multimodal Jailbreaking of Vision-Language and Audio Models through Perceptually Simple Transformations
  5. 2025Transferability of Adversarial Attacks in Video-based MLLMs: A Cross-modal Image-to-Video Approach
  6. 2026Visual Memory Injection Attacks for Multi-Turn Conversations
  7. 2026Adversarial Attacks on Multimodal Large Language Models: A Comprehensive Survey
  8. 2024Enhancing Cross-Prompt Transferability in Vision-Language Models through Contextual Injection of Target Tokens