mlsanityData to 7 Oct 2026

Visual Memory Attacks Can Persist Through The KV Cache

arXiv 2610.09027 ↗First posted 6 Oct 2026cs.CR cs.AI cs.CVcited by 0

New in Efficient LLM inference and serving; no direction yet. Efficient LLM inference and serving

Abstract (arXiv)

Modern language model systems operate autonomously over increasingly long contexts containing untrusted text and images. Can an adversarial input continue to steer a model even after that input is removed from its context? We show that attacks can be trained to persist through the key/value (KV) cache of subsequent tokens, allowing adversarial influence to outlive direct access to its source.We consider the Visual Memory Injection (VMI; Schlarmann and Hein, 2026) attack setting, in which an adversarial image that stays in the context plants a hidden backdoor: the model behaves normally until a chosen trigger elicits an attacker-chosen response. We first demonstrate persistence in this setting with optimized soft prompts, which remain effective after we mask the prompt from attention. We then introduce Persistent Visual Memory Injection (P-VMI), which optimizes images to preserve this adversarial behaviour after they are masked from attention. These attacks persist over conversations substantially longer than those used during optimization. On Qwen3-VL-8B-Instruct, P-VMI achieves up to approximately 90% target success in its strongest configuration and remains effective under a stricter removal setting that exposes the image only on the first turn. A cache-swap ablation localizes the persistent influence to the KV cache. Finally, we show that these attacks can be trained to survive compaction that retains the KV cache of a summary generated by the same model, demonstrating that adversarial behaviour can persist in cached state without continued access to its source.

Key facts. No citing papers in the map yet (data through 7 Oct 2026). Builds on 42 of its 43 references that are in the map. First posted on arXiv 6 Oct 2026.

Builds on

42 of 43 references.

References · 42

  1. 2017Attention Is All You Needcited by 59,932
  2. 2014Explaining and Harnessing Adversarial Examplescited by 7,287
  3. 2017Towards Deep Learning Models Resistant to Adversarial Attackscited by 6,326
  4. 2013Intriguing properties of neural networkscited by 5,114
  5. 2021The Power of Scale for Parameter-Efficient Prompt Tuningcited by 3,769
  6. 2016Towards Evaluating the Robustness of Neural Networkscited by 2,976
  7. 2023Universal and Transferable Adversarial Attacks on Aligned Language Modelscited by 2,914
  8. 2025Qwen3-VL Technical Reportcited by 2,534
  9. 2023Efficient Streaming Language Models with Attention Sinkscited by 2,115
  10. 2023Jailbroken: How Does LLM Safety Training Fail?cited by 1,565
  11. 2020Reliable evaluation of adversarial robustness with an ensemble of diverse parameter-free attackscited by 1,442
  12. 2023Not what you've signed up for: Compromising Real-World LLM-Integrated Applications with Indirect Prompt Injectioncited by 1,339
30 more
  1. 2023MemGPT: Towards LLMs as Operating Systemscited by 1,295
  2. 2018Obfuscated Gradients Give a False Sense of Security: Circumventing Defenses to Adversarial Examplescited by 1,259
  3. 2024Refusal in Language Models Is Mediated by a Single Directioncited by 960
  4. 2024SnapKV: LLM Knows What You are Looking for Before Generationcited by 848
  5. 2023H2O: Heavy-Hitter Oracle for Efficient Generative Inference of Large Language Modelscited by 805
  6. 2024An Image is Worth 1/2 Tokens After Layer 2: Plug-and-Play Inference Acceleration for Large Vision-Language Modelscited by 629
  7. 2023Baseline Defenses for Adversarial Attacks Against Aligned Language Modelscited by 608
  8. 2020On Adaptive Attacks to Adversarial Example Defensescited by 547
  9. 2024Sleeper Agents: Training Deceptive LLMs that Persist Through Safety Trainingcited by 478
  10. 2024AgentPoison: Red-teaming LLM Agents via Poisoning Memory or Knowledge Basescited by 402
  11. 2023Visual Adversarial Examples Jailbreak Aligned Large Language Modelscited by 307
  12. 2023Are aligned neural networks adversarially aligned?cited by 305
  13. 2023Jailbreak in pieces: Compositional Adversarial Attacks on Multi-Modal Language Modelscited by 260
  14. 2026Gemma 4 Technical Reportcited by 187
  15. 2025Constitutional Classifiers: Defending against Universal Jailbreaks across Thousands of Hours of Red Teamingcited by 172
  16. 2023Image Hijacks: Adversarial Images can Control Generative Models at Runtimecited by 165
  17. 2023On the Adversarial Robustness of Multi-Modal Foundation Modelscited by 142
  18. 2024Soft Prompt Threats: Attacking Safety Alignment and Unlearning in Open-Source LLMs through the Embedding Spacecited by 88
  19. 2025Cartridges: Lightweight and general-purpose long context representations via self-studycited by 74
  20. 2023Recursively Summarizing Enables Long-Term Dialogue Memory in Large Language Modelscited by 62
  21. 2026Zombie Agents: Persistent Control of Self-Evolving LLM Agents via Self-Reinforcing Injectionscited by 44
  22. 2026Fast KV Compaction via Attention Matchingcited by 28
  23. 2026MEMENTO: Teaching LLMs to Manage Their Own Contextcited by 16
  24. 2025MIP against Agent: Malicious Image Patches Hijacking Multimodal OS Agentscited by 12
  25. 2025AirCache: Activating Inter-modal Relevancy KV Cache Compression for Efficient Large Vision-Language Model Inferencecited by 4
  26. 2026Visual Memory Injection Attacks for Multi-Turn Conversationscited by 2
  27. 2026TGV-KV: Text-Grounded KV Eviction for Vision-Language Modelscited by 1
  28. 2026Do Agents Dream of False Memories? Black-box Visual Attacks on Long-term Memory in Multimodal AI Agentscited by 1
  29. 2026HijackKV: New Threat in Position-Independent KV Cache Reusecited by 1
  30. 2026KV-streams for Efficient Compaction in Agentic Reinforcement Learningcited by 1

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2026Hidden in Memory: Sleeper Memory Poisoning in LLM Agents
  2. 2025Dynamic Token Reweighting for Robust Vision-Language Models
  3. 2024Break the Visual Perception: Adversarial Attacks Targeting Encoded Visual Tokens of Large Vision-Language Models
  4. 2026Render Before Reading: Visual Rendering as a Prompt Injection Defense
  5. 2024Safeguarding Vision-Language Models Against Patched Visual Prompt Injectors
  6. 2025Transferable Adversarial Attacks on Black-Box Vision-Language Models
  7. 2026Repeat-After-Me: Black-Box Adaptive Visual Prompt Injection
  8. 2026VisInject: Disruption != Injection -- A Dual-Dimension Evaluation of Universal Adversarial Attacks on Vision-Language Models