# Trojan Hippo: Weaponizing Agent Memory for Data Exfiltration arXiv: https://arxiv.org/abs/2605.01970 ยท HTML: https://mlsanity.com/paper/2605.01970/ ## Key facts - Cited by 12 papers in the map (data through 14 Oct 2026). - Placed in Agent threats, guardrails and governance, one of 16 directions in LLM agents and tool use. - Builds on 21 of its 60 references that are in the map. - First posted on arXiv 3 May 2026. ## Placed in - LLM agents and tool use / [Agent threats, guardrails and governance](https://mlsanity.com/agents/agent-threats-guardrails-and-governance/) ## Builds on (most cited first, 21 of 21) - [Generative Agents: Interactive Simulacra of Human Behavior](https://mlsanity.com/paper/2304.03442/) (2023) - [Not what you've signed up for: Compromising Real-World LLM-Integrated Applications with Indirect Prompt Injection](https://mlsanity.com/paper/2302.12173/) (2023) - [MemGPT: Towards LLMs as Operating Systems](https://mlsanity.com/paper/2310.08560/) (2023) - [Evaluating Very Long-Term Conversational Memory of LLM Agents](https://mlsanity.com/paper/2402.17753/) (2024) - [Tree of Attacks: Jailbreaking Black-Box LLMs Automatically](https://mlsanity.com/paper/2312.02119/) (2023) - [The Instruction Hierarchy: Training LLMs to Prioritize Privileged Instructions](https://mlsanity.com/paper/2404.13208/) (2024) - [AgentPoison: Red-teaming LLM Agents via Poisoning Memory or Knowledge Bases](https://mlsanity.com/paper/2407.12784/) (2024) - [Memory in the Age of AI Agents](https://mlsanity.com/paper/2512.13564/) (2025) - [Defeating Prompt Injections by Design](https://mlsanity.com/paper/2503.18813/) (2025) - [Defending Against Indirect Prompt Injection Attacks With Spotlighting](https://mlsanity.com/paper/2403.14720/) (2024) - [Evaluating Memory in LLM Agents via Incremental Multi-Turn Interactions](https://mlsanity.com/paper/2507.05257/) (2025) - [SecAlign: Defending Against Prompt Injection with Preference Optimization](https://mlsanity.com/paper/2410.05451/) (2024) - [Memory Injection Attacks on LLM Agents via Query-Only Interaction](https://mlsanity.com/paper/2503.03704/) (2025) - [Progent: Securing AI Agents with Privilege Control](https://mlsanity.com/paper/2504.11703/) (2025) - [A-MemGuard: A Proactive Defense Framework for LLM-Based Agent Memory](https://mlsanity.com/paper/2510.02373/) (2025) - [AgentVigil: Generic Black-Box Red-teaming for Indirect Prompt Injection against LLM Agents](https://mlsanity.com/paper/2505.05849/) (2025) - [Securing Agentic AI: A Comprehensive Threat Model and Mitigation Framework for Generative AI Agents](https://mlsanity.com/paper/2504.19956/) (2025) - [Poison Once, Exploit Forever: Environment-Injected Memory Poisoning Attacks on Web Agents](https://mlsanity.com/paper/2604.02623/) (2026) - [CommandSans: Securing AI Agents with Surgical Precision Prompt Sanitization](https://mlsanity.com/paper/2510.08829/) (2025) - [Securing AI Agents Against Prompt Injection Attacks](https://mlsanity.com/paper/2511.15759/) (2025) - [Exploiting Web Search Tools of AI Agents for Data Exfiltration](https://mlsanity.com/paper/2510.09093/) (2025) ## Built on by (most cited first, 12 of 12) - [Securing LLM-Agent Long-Term Memory Against Poisoning: Non-Malleable, Origin-Bound Authority with Machine-Checked Guarantees](https://mlsanity.com/paper/2606.24322/) (2026) - [Safeguarding LLM Agents against Long-Horizon Threats via Shadow Memory](https://mlsanity.com/paper/2605.03228/) (2026) - [Injection-Execution Dissociation: A Mechanistic Evaluation of Persistent Memory Attacks and Defenses in Stateful LLM Agents](https://mlsanity.com/paper/2605.08442/) (2026) - [Deployment-Time Memorization in Foundation-Model Agents](https://mlsanity.com/paper/2606.10062/) (2026) - [MemSecBench: Tracking Agent Memory Poisoning from Persistence to Consequence and Repair](https://mlsanity.com/paper/2607.27080/) (2026) - [HarnessSafe: Evaluating Safety Across Persistent Carriers in Agent Harnesses](https://mlsanity.com/paper/2608.06984/) (2026) - [Stateful Agent Backdoor](https://mlsanity.com/paper/2605.06158/) (2026) - [Isolation as a First-Class Principle for LLM-Agent System Safety: Concepts, Taxonomy, Challenges and Future Directions](https://mlsanity.com/paper/2607.12406/) (2026) - [MemSentry: A Framework for Detecting Persistent Memory Poisoning in Agentic AI](https://mlsanity.com/paper/2609.08747/) (2026) - [Reflections on Trusting Trust, Revisited: Contaminating Self-Modifying AI Coding Agents with Poisoned Benchmarks](https://mlsanity.com/paper/2609.17817/) (2026) - [Beyond Predictable Paths: Redefining AI Security Incident Reporting for Agents](https://mlsanity.com/paper/2609.24515/) (2026) - [Share-Borne AI Virus: Memory-Hopping Attacks Across LLM Agents](https://mlsanity.com/paper/2609.35576/) (2026) ## Abstract (arXiv) Memory systems enable otherwise stateless LLM agents to persist user information across sessions, but also introduce a new attack surface. The Trojan Hippo attack is a class of persistent memory attacks that operates under a more realistic threat model than prior memory poisoning work. The attacker plants a dormant payload into an agent's long-term memory via a single untrusted tool call (e.g., a crafted email), which activates only when the user later discusses sensitive topics such as finance, health, or identity, and exfiltrates high-value personal data to the attacker. While anecdotal demonstrations of such attacks have appeared against deployed systems, no prior work systematically evaluates them across heterogeneous memory architectures and defenses. We introduce Trojan Hippo Bench, a dynamic evaluation framework for persistent memory attacks and memory-layer defenses, comprising two components, (1) an OpenEvolve-based adaptive red-teaming benchmark that stress-tests defenses and memory backends against continuously refined attacks, and (2) a capability-aware security-utility analysis for persistent memory systems, enabling principled reasoning about defense deployment across different usage profiles. Instantiated on an email assistant across four memory backends (explicit tool memory, agentic memory, RAG, and sliding-window context), the undefended Trojan Hippo attack achieves up to 85-100% attack success rate (ASR) against current frontier models from OpenAI and Google, with planted memories successfully activating even after 100 benign sessions. We evaluate four memory-system defenses inspired by basic security principles; they reduce attack success to as low as 0-5%, but at utility costs that vary widely with the task mix. Effective real-world deployment thus remains an open challenge that Trojan Hippo Bench is built to study; we release it to support reuse and extension.