mlsanityData to 7 Oct 2026

From Attack Success to Attack Severity: Counterfactual Memory Attacks on LLM Agents

arXiv 2609.34132 ↗First posted 28 Sep 2026cs.AIcited by 0

On the map: LLM agents and tool use → Agent memory systems also fits: Agent threats, guardrails and governance

Abstract (arXiv)

As LLM agents increasingly rely on persistent memory for long-horizon and personalized behavior, they can retain and reuse information across interactions, but this also creates a lasting channel through which malicious memory writes can influence future behavior. Persistent-memory attacks are typically evaluated by whether they succeed, yet successful attacks can leave persistent states with substantially different downstream consequences. We study this severity as a distinct attack-design objective and formalize it with counterfactual memory regret (CMR), the paired increase in expected downstream loss relative to clean memory. We introduce MemHarm, which predeclares a finite class of sparse, grounded semantic edits, evaluates candidates through the normal agent memory interface using offline paired-loss feedback, and certifies resolved selections within that class. Compared with attack-success optimization, CMR-guided selection produces substantially larger downstream loss while retaining most of the success-rate gain. Across two agent benchmarks and diverse memory designs, MemHarm attains the highest CMR point estimates among the evaluated general attacks on identical support. Factor-removal interventions link this harm to the selected semantic factor, and native-agent deployments verify the write-to-fresh-process attack path.

Key facts. No citing papers in the map yet (data through 7 Oct 2026). Placed in Agent memory systems, one of 16 directions in LLM agents and tool use. Builds on 33 of its 48 references that are in the map. First posted on arXiv 28 Sep 2026.

Builds on

33 of 48 references.

References · 33

  1. 2023Reflexion: Language Agents with Verbal Reinforcement Learningcited by 4,064
  2. 2023Generative Agents: Interactive Simulacra of Human Behaviorcited by 3,500
  3. 2023Universal and Transferable Adversarial Attacks on Aligned Language Modelscited by 2,914
  4. 2023Voyager: An Open-Ended Embodied Agent with Large Language Modelscited by 2,081
  5. 2023MemGPT: Towards LLMs as Operating Systemscited by 1,295
  6. 2024τ-bench: A Benchmark for Tool-Agent-User Interaction in Real-World Domainscited by 1,280
  7. 2017Counterfactual Explanations without Opening the Black Box: Automated Decisions and the GDPRcited by 1,086
  8. 2019Explaining Machine Learning Classifiers through Diverse Counterfactual Explanationscited by 612
  9. 2025τ2-Bench: Evaluating Conversational Agents in a Dual-Control Environmentcited by 558
  10. 2024AgentPoison: Red-teaming LLM Agents via Poisoning Memory or Knowledge Basescited by 402
  11. 2024PoisonedRAG: Knowledge Corruption Attacks to Retrieval-Augmented Generation of Large Language Modelscited by 251
  12. 2020Algorithmic Recourse: from Counterfactual Explanations to Interventionscited by 250
21 more
  1. 2015Non-stochastic Best Arm Identification and Hyperparameter Optimizationcited by 219
  2. 2020Estimating means of bounded random variables by bettingcited by 164
  3. 2024Agent Smith: A Single Image Can Jailbreak One Million Multimodal LLM Agents Exponentially Fastcited by 149
  4. 2025Memory Injection Attacks on LLM Agents via Query-Only Interactioncited by 127
  5. 2024Machine Against the RAG: Jamming Retrieval-Augmented Generation with Blocker Documentscited by 93
  6. 2025MemoryGraft: Persistent Compromise of LLM Agents via Poisoned Experience Retrievalcited by 65
  7. 2026Memory Poisoning Attack and Defense on Memory Based LLM-Agentscited by 40
  8. 2025Real AI Agents with Fake Memories: Fatal Context Manipulation Attacks on Web3 Agentscited by 23
  9. 2026From Untrusted Input to Trusted Memory: A Systematic Study of Memory Poisoning Attacks in LLM Agentscited by 23
  10. 2026Hidden in Memory: Sleeper Memory Poisoning in LLM Agentscited by 22
  11. 2025SABER: Small Actions, Big Errors -- Safeguarding Mutating Steps in LLM Agentscited by 21
  12. 2026A Survey on Long-Term Memory Security in LLM Agents: Attacks, Defenses, and Governance Across the Memory Lifecyclecited by 21
  13. 2026Poison Once, Exploit Forever: Environment-Injected Memory Poisoning Attacks on Web Agentscited by 18
  14. 2025CPA-RAG:Covert Poisoning Attacks on Retrieval-Augmented Generation in Large Language Modelscited by 17
  15. 2025DrunkAgent: Stealthy Memory Corruption in LLM-Powered Recommender Agentscited by 12
  16. 2026MemAudit: Post-hoc Auditing of Poisoned Agent Memory via Causal Attribution and Structural Anomaly Detectioncited by 6
  17. 2026Injection-Execution Dissociation: A Mechanistic Evaluation of Persistent Memory Attacks and Defenses in Stateful LLM Agentscited by 5
  18. 2026Bad Memory: Evaluating Prompt Injection Risks from Memory in Agentic Systemscited by 5
  19. 2026Remembering More, Risking More: Longitudinal Safety Risks in Memory-Equipped LLM Agentscited by 4
  20. 2026InjecMEM: Memory Injection Attack on LLM Agent Memory Systemscited by 3
  21. 2026SilentRetrieval: Hijacking Retrieval-Augmented Generation via Semantically-Preserving Adversarial Data Poisoningcited by 1

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2026MemEvoBench: Benchmarking Safety Risks from Memory Misevolution in LLM Agents
  2. 2026MemPoison: Uncovering Persistent Memory Threats and Structural Blind Spots in LLM Agents
  3. 2025A-MemGuard: A Proactive Defense Framework for LLM-Based Agent Memory
  4. 2026From Storage to Steering: Memory Control Flow Attacks on LLM Agents
  5. 2026Understanding Stage-Wise Utility-Risk Trade-offs in LLM Agent Memory
  6. 2026MemSentry: A Framework for Detecting Persistent Memory Poisoning in Agentic AI
  7. 2026SMSR: Certified Defence Against Runtime Memory Poisoning in Persistent LLM Agent Systems
  8. 2026MemSecBench: Tracking Agent Memory Poisoning from Persistence to Consequence and Repair