mlsanityData to 14 Oct 2026

Unveiling Privacy Risks in LLM Agent Memory

arXiv 2502.13172 ↗First posted 17 Feb 2025cs.CR cs.AIcited by 101

On the map: LLM agents and tool use → Agent threats, guardrails and governance

Abstract (arXiv)

Large Language Model (LLM) agents have become increasingly prevalent across various real-world applications. They enhance decision-making by storing private user-agent interactions in the memory module for demonstrations, introducing new privacy risks for LLM agents. In this work, we systematically investigate the vulnerability of LLM agents to our proposed Memory EXTRaction Attack (MEXTRA) under a black-box setting. To extract private information from memory, we propose an effective attacking prompt design and an automated prompt generation method based on different levels of knowledge about the LLM agent. Experiments on two representative agents demonstrate the effectiveness of MEXTRA. Moreover, we explore key factors influencing memory leakage from both the agent designer's and the attacker's perspectives. Our findings highlight the urgent need for effective memory safeguards in LLM agent design and deployment.

Key facts. Cited by 101 papers in the map (data through 14 Oct 2026). Placed in Agent threats, guardrails and governance, one of 16 directions in LLM agents and tool use. Builds on 37 of its 50 references that are in the map. First posted on arXiv 17 Feb 2025.

Builds on

37 of 50 references.

Closest ancestors · 4

Same direction, newest first

  1. 2024Breaking Agents: Compromising Autonomous LLM Agents Through Malfunction Amplification30 Jul
  2. 2024The Emerged Security and Privacy of LLM Agent: A Survey with Case Studies28 Jul
  3. 2024InjecAgent: Benchmarking Indirect Prompt Injections in Tool-Integrated Large Language Model Agents5 Mar
  4. 2024WIPI: A New Web Threat for LLM-Driven Web Agents26 Feb

Other references · 33

  1. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,399
  2. 2024The Llama 3 Herd of Modelscited by 14,715
  3. 2019RoBERTa: A Robustly Optimized BERT Pretraining Approachcited by 13,372
  4. 2020Retrieval-Augmented Generation for Knowledge-Intensive NLP Taskscited by 9,297
  5. 2019Sentence-BERT: Sentence Embeddings using Siamese BERT-Networkscited by 9,170
  6. 2022ReAct: Synergizing Reasoning and Acting in Language Modelscited by 8,695
  7. 2023Universal and Transferable Adversarial Attacks on Aligned Language Modelscited by 3,104
  8. 2023A Survey on Large Language Model based Autonomous Agentscited by 2,242
  9. 2020MiniLM: Deep Self-Attention Distillation for Task-Agnostic Compression of Pre-Trained Transformerscited by 1,463
  10. 2023The Rise and Potential of Large Language Model Based Agents: A Surveycited by 1,439
  11. 2022WebShop: Towards Scalable Real-World Web Interaction with Grounded Language Agentscited by 1,251
  12. 2023Mind2Web: Towards a Generalist Agent for the Webcited by 1,242
21 more
  1. 2023Llama Guard: LLM-based Input-Output Safeguard for Human-AI Conversationscited by 1,207
  2. 2020MPNet: Masked and Permuted Pre-training for Language Understandingcited by 981
  3. 2022Ignore Previous Prompt: Attack Techniques For Language Modelscited by 832
  4. 2023Baseline Defenses for Adversarial Attacks Against Aligned Language Modelscited by 651
  5. 2024A Survey on the Memory Mechanism of Large Language Model based Agentscited by 634
  6. 2023Prompt Injection attack against LLM-integrated Applicationscited by 605
  7. 2024GPT-4V(ision) is a Generalist Web Agent, if Groundedcited by 563
  8. 2024AppWorld: A Controllable World of Apps and People for Benchmarking Interactive Coding Agentscited by 401
  9. 2023A Language Agent for Autonomous Drivingcited by 164
  10. 2024The Good and The Bad: Exploring Privacy Issues in Retrieval-Augmented Generation (RAG)cited by 159
  11. 2023DeID-GPT: Zero-shot Medical Text De-Identification by GPT-4cited by 130
  12. 2024PLeak: Prompt Leaking Attacks against Large Language Model Applicationscited by 127
  13. 2023Conversational Health Agents: A Personalized LLM-Powered Agent Frameworkcited by 92
  14. 2024RAP: Retrieval-Augmented Planning with Contextual Memory for Multimodal LLM Agentscited by 80
  15. 2024If LLM Is the Wizard, Then Code Is the Wand: A Survey on How Code Empowers Large Language Models to Serve as Intelligent Agentscited by 79
  16. 2024Follow My Instruction and Spill the Beans: Scalable Data Extraction from Retrieval-Augmented Generation Systemscited by 76
  17. 2023Evaluating the Instruction-Following Robustness of Large Language Models to Prompt Injectioncited by 59
  18. 2023Personalized Autonomous Driving with Large Language Models: Field Experimentscited by 48
  19. 2024Feedback-Guided Extraction of Knowledge Base from Retrieval-Augmented LLM Applicationscited by 37
  20. 2024Reinforcement Learning for Optimizing RAG for Domain Chatbotscited by 26
  21. 2024Pirates of the RAG: Adaptively Attacking LLMs to Leak Knowledge Basescited by 18

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2025: 28; 2026: 73.

101 papers.

Sort:

By area:

  1. 2025Memory in the Age of AI Agentscited by 303
  2. 2025Large Language Model Agent: A Survey on Methodology, Applications and Challengescited by 205
  3. 2025A Survey of Self-Evolving Agents: What, When, How, and Where to Evolve on the Path to Artificial Super Intelligencecited by 170
  4. 2025A Survey of WebAgents: Towards Next-Generation AI Agents for Web Automation with Large Foundation Modelscited by 87
  5. 2025A-MemGuard: A Proactive Defense Framework for LLM-Based Agent Memorycited by 67
  6. 2025A Survey of LLM-Driven AI Agent Communication: Protocols, Security Risks, and Defense Countermeasurescited by 57
  7. 2026From Agent Traces to Trust: A Survey of Evidence Tracing and Execution Provenance in LLM Agentscited by 37
  8. 2025LLM Agents Should Employ Security Principlescited by 29
  9. 2025A Survey on Autonomy-Induced Security Risks in Large Model-Based Agentscited by 29
  10. 2025MAGPIE: A benchmark for Multi-AGent contextual PrIvacy Evaluationcited by 29
  11. 2026A Survey on Long-Term Memory Security in LLM Agents: Attacks, Defenses, and Governance Across the Memory Lifecyclecited by 27
  12. 2025A Survey on the Safety and Security Threats of Computer-Using Agents: JARVIS or Ultron?cited by 23
38 more
  1. 2026From Storage to Steering: Memory Control Flow Attacks on LLM Agentscited by 19
  2. 2026Governing Evolving Memory in LLM Agents: Risks, Mechanisms, and the Stability and Safety Governed Memory (SSGM) Frameworkcited by 18
  3. 2026SafeHarness: Lifecycle-Integrated Security Architecture for LLM-based Agent Deploymentcited by 15
  4. 2025A Survey on Agent Workflow -- Status and Futurecited by 14
  5. 2026Governed Shared Memory for Multi-Agent LLM Systemscited by 13
  6. 2025Topology Matters: Measuring Memory Leakage in Multi-Agent LLMscited by 11
  7. 2025Agent Tools Orchestration Leaks More: Dataset, Benchmark, and Mitigationcited by 11
  8. 2026A Systematic Survey of Security Threats and Defenses in LLM-Based AI Agents: A Layered Attack Surface Frameworkcited by 11
  9. 2026Your LLM Agent Can Leak Your Data: Data Exfiltration via Backdoored Tool Usecited by 10
  10. 2025LLM Applications: Current Paradigms and the Next Frontiercited by 8
  11. 2025Privacy-Aware Decoding: Mitigating Privacy Leakage of Large Language Models in Retrieval-Augmented Generationcited by 8
  12. 2025Meta-Policy Reflexion: Reusable Reflective Memory and Rule Admissibility for Resource-Efficient LLM Agentcited by 7
  13. 2026MemPrivacy: Privacy-Preserving Personalized Memory Management for Edge-Cloud Agentscited by 7
  14. 2026Remembering More, Risking More: Longitudinal Safety Risks in Memory-Equipped LLM Agentscited by 7
  15. 2026Toward Secure LLM Agents: Threat Surfaces, Attacks, Defenses, and Evaluationcited by 7
  16. 2025Agentic Services Computingcited by 6
  17. 2026INFA-Guard: Mitigating Malicious Propagation via Infection-Aware Safeguarding in LLM-Based Multi-Agent Systemscited by 6
  18. 2026AgentRaft: Automated Detection of Data Over-Exposure in LLM Agentscited by 6
  19. 2025Beyond Text: Unveiling Privacy Vulnerabilities in Multi-modal Retrieval-Augmented Generationcited by 5
  20. 2025Network-Level Prompt and Trait Leakage in Local Research Agentscited by 5
  21. 2025PEAR: Planner-Executor Agent Robustness Benchmarkcited by 5
  22. 2025Don't Trust Your Upstream: Exploiting LLM Multi-Agent System via Topology-Guided Adversarial Propagationcited by 5
  23. 2026Long Context, Less Focus: A Scaling Gap in LLMs Revealed through Privacy and Personalizationcited by 5
  24. 2026ADAM: A Systematic Data Extraction Attack on Agent Memory via Adaptive Queryingcited by 5
  25. 2026Geometry-Calibrated Conformal Abstention for Language Modelscited by 5
  26. 2026State Contamination in Memory-Augmented LLM Agentscited by 5
  27. 2025Past, Present, and Future of Bug Tracking in the Generative AI Eracited by 4
  28. 2025Chain-of-Trigger: An Agentic Backdoor that Paradoxically Enhances Agentic Robustnesscited by 4
  29. 2026Deployment-Time Memorization in Foundation-Model Agentscited by 4
  30. 2026FragFuse: Bypassing Access Control of Large Language Model Agents via Memory-Based Query Fragmentation and Fusioncited by 4
  31. 2026Agent Skills Matter: Inferring Proprietary Skills from Execution Trajectoriescited by 4
  32. 2026Self-Evolving Agents as Dynamic Graph Transformation: A Survey and New Perspectivecited by 4
  33. 2026MemGuard: Persisting Verifier Signals for LLM-Agent Memory Governancecited by 4
  34. 2026InjecMEM: Memory Injection Attack on LLM Agent Memory Systemscited by 4
  35. 2026Query-Efficient Agentic Graph Extraction Attacks on GraphRAG Systemscited by 3
  36. 2025The Role of Open-Source LLMs in Shaping the Future of GeoAIcited by 2
  37. 2025Empowering Real-World: A Survey on the Technology, Practice, and Evaluation of LLM-driven Industry Agentscited by 2
  38. 2025A Longitudinal Measurement of Privacy Policy Evolution for Large Language Modelscited by 2

50 most cited of 101 All 101 (JSON)

Similar, not cited

Close in wording, no citation either way

  1. 2025Commercial LLM Agents Are Already Vulnerable to Simple Yet Dangerous Attacks
  2. 2026MemEvoBench: Benchmarking Safety Risks from Memory Misevolution in LLM Agents
  3. 2024Imprompter: Tricking LLM Agents into Improper Tool Use
  4. 2025Memory Injection Attacks on LLM Agents via Query-Only Interaction
  5. 2024Whispers in the Machine: Confidentiality in Agentic Systems
  6. 2026From Untrusted Input to Trusted Memory: A Systematic Study of Memory Poisoning Attacks in LLM Agents
  7. 2026Safeguarding LLM Agents against Long-Horizon Threats via Shadow Memory
  8. 2026Hijacking Agent Memory: Stealthy Trojan Attacks Through Conversational Interaction