mlsanityData to 7 Oct 2026

Repeat-After-Me: Black-Box Adaptive Visual Prompt Injection

arXiv 2609.04533 ↗First posted 3 Sep 2026cs.CR cs.AIcited by 0

On the map: LLM safety and security → Prompt injection and agent security

Abstract (arXiv)

Prompt injection is widely recognized as a major security threat to AI agents that interact with untrusted external data, such as websites, documents, and emails. Prior work has shown that, in the text domain, black-box prompt injection can achieve near-perfect attack success rates (ASRs). In the image domain, however, existing visual prompt injection methods are substantially less effective in attacking frontier commercial VLMs for materially harmful behavior. Achieving such outputs is hard because it requires a long and/or format-compliant target string, such as a precise, parseable native tool call with exact function names and arguments. We present Repeat-After-Me, a black-box adaptive visual prompt injection attack that can reveal personally identifiable information or make malicious tool calls. Across both open-weight and commercial frontier VLMs, including Qwen3.6-27B and GPT-5.5, our method achieves ASRs exceeding 82% and 47%, respectively, under a realistic setting in which the benign user prompt is semantically unrelated to the injected task and does not verbally authorize it. Our optimized injection has non-trivial attack transferability across commercial VLMs and benign samples. We show our attack works in cases where adaptive textual prompt injection fails. In a real-world OpenClaw agent connected to Discord, an untrusted user can use a minimally injected image from our attack to overwrite TOOLS.md, enabling future sensitive behaviors like remote code execution and secret exfiltration. We discuss potential defenses.

Key facts. No citing papers in the map yet (data through 7 Oct 2026). Placed in Prompt injection and agent security, one of 12 directions in LLM safety and security. Builds on 24 of its 41 references that are in the map. First posted on arXiv 3 Sep 2026.

Builds on

24 of 41 references.

Closest ancestors · 3

Same direction, newest first

  1. 2025ARGUS: Defending Against Multimodal Indirect Prompt Injection via Steering Instruction-Following Behavior5 Dec
  2. 2025EchoLeak: The First Real-World Zero-Click Prompt Injection Exploit in a Production LLM System6 Sep
  3. 2025VPI-Bench: Visual Prompt Injection Attacks for Computer-Use Agents3 Jun

Other references · 21

  1. 2023Jailbreaking Black Box Large Language Models in Twenty Queriescited by 1,326
  2. 2020DocVQA: A Dataset for VQA on Document Imagescited by 1,288
  3. 2023Large Language Models as Optimizerscited by 830
  4. 2023Tree of Attacks: Jailbreaking Black-Box LLMs Automaticallycited by 619
  5. 2023FigStep: Jailbreaking Large Vision-Language Models via Typographic Visual Promptscited by 368
  6. 2023Visual Adversarial Examples Jailbreak Aligned Large Language Modelscited by 307
  7. 2023Are aligned neural networks adversarially aligned?cited by 305
  8. 2023Image Hijacks: Adversarial Images can Control Generative Models at Runtimecited by 165
  9. 2024Images are Achilles' Heel of Alignment: Exploiting Visual Vulnerabilities for Jailbreaking Multimodal Large Language Modelscited by 121
  10. 2024PLeak: Prompt Leaking Attacks against Large Language Model Applicationscited by 113
  11. 2024Attacking Vision-Language Computer Agents via Pop-upscited by 112
  12. 2024Jailbreak Vision Language Models via Bi-Modal Adversarial Promptcited by 103
9 more
  1. 2023Effective Prompt Extraction from Language Modelscited by 83
  2. 2025Manipulating Multimodal Agents via Cross-Modal Prompt Injectioncited by 42
  3. 2024IDEATOR: Jailbreaking and Benchmarking Large Vision-Language Models Using Themselvescited by 34
  4. 2025CIMemories: A Compositional Benchmark for Contextual Integrity of Persistent Memory in LLMscited by 22
  5. 2025JPS: Jailbreak Multimodal Large Language Models with Collaborative Visual Perturbation and Textual Steeringcited by 10
  6. 2026Image-based Prompt Injection: Hijacking Multimodal LLMs through Visually Embedded Adversarial Instructionscited by 8
  7. 2024Self-interpreting Adversarial Imagescited by 7
  8. 2025Odysseus: Jailbreaking Commercial Multimodal LLM-integrated Systems via Dual Steganographycited by 6
  9. 2025Adversarial Confusion Attack: Disrupting Multimodal Large Language Modelscited by 3

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2026Covert Visual Prompt Injection against Commercial Multimodal Large Language Models
  2. 2026Defense Against Indirect Prompt Injection via Tool Result Parsing
  3. 2026Assessing Automated Prompt Injection Attacks in Agentic Environments
  4. 2026When AUC 0.998 Is Not Enough: A Candidate Evaluation Protocol for Hidden-State Probes of Indirect Prompt Injection in Multimodal Computer-Use Agents
  5. 2026Defending against Adaptive Prompt Injection Attacks via Reasoning-enabled Task Alignment
  6. 2024Empirical Analysis of Large Vision-Language Models against Goal Hijacking via Visual Prompt Injection
  7. 2025AgentTypo: Adaptive Typographic Prompt Injection Attacks against Black-box Multimodal Agents
  8. 2026Physical Prompt Injection Attacks on Large Vision-Language Models