Deployment-Time Memorization in Foundation-Model Agents
On the map: LLM agents and tool use → Agent memory systems
Abstract (arXiv)
Foundation-model agents are increasingly long-lived systems that remember users across interactions, making memorization an explicit deployment-time function rather than solely a property of model weights. Existing work addresses parametric memorization or audits fixed memory configurations, but does not characterize how memory-design choices jointly shape personalization utility, extraction risk, and deletion fidelity. We study this surface as deployment-time memorization, formulating agent memory as a privacy-utility frontier measured by Personalization Recall (PR) and Adversarial Extraction Rate (AER), and sweeping three memory-design knobs: summarization aggressiveness, retrieval breadth (k), and deletion mode. We further introduce the Forgetting Residue Score (FRS) to quantify whether deleted information remains recoverable from derived memory tiers. On LongMemEval, key-fact summarization reduces canary extraction by 76% on Gemma 3 12B and 64% on GPT-4o-mini while preserving nearly all personalization recall; critically, once content is compressed away, increasing k no longer restores leakage. The same compression, however, induces a deletion-fidelity failure: raw-only deletion leaves derived summary copies recoverable in approximately 20% of instances, and only full-pipeline purge or tombstone redaction drives worst-tier residue to zero. Together, these results establish that persistent agent memory must be evaluated as a first-class memorization mechanism -- assessed by what it helps agents recall, what it makes extractable, and what it can truly erase.
Key facts. Cited by 4 papers in the map (data through 14 Oct 2026). Placed in Agent memory systems, one of 16 directions in LLM agents and tool use. Builds on 16 of its 21 references that are in the map. First posted on arXiv 8 Jun 2026.
Builds on
16 of 21 references.
Closest ancestors · 1
Same direction, newest first
Other references · 15
- 2020Retrieval-Augmented Generation for Knowledge-Intensive NLP Taskscited by 9,297
- 2019Sentence-BERT: Sentence Embeddings using Siamese BERT-Networkscited by 9,170
- 2023Generative Agents: Interactive Simulacra of Human Behaviorcited by 3,568
- 2016Deep Learning with Differential Privacycited by 2,728
- 2020Extracting Training Data from Large Language Modelscited by 2,189
- 2023Not what you've signed up for: Compromising Real-World LLM-Integrated Applications with Indirect Prompt Injectioncited by 1,443
- 2019Machine Unlearningcited by 1,061
- 2018The Secret Sharer: Evaluating and Testing Unintended Memorization in Neural Networkscited by 864
- 2022Quantifying Memorization Across Neural Language Modelscited by 763
- 2024LongMemEval: Benchmarking Chat Assistants on Long-Term Interactive Memorycited by 707
- 2023Scalable Extraction of Training Data from (Production) Language Modelscited by 434
- 2023Can LLMs Keep a Secret? Testing Privacy Implications of Language Models via Contextual Integrity Theorycited by 230
Built on by
Citing papers per year: 2026: 4.
4 papers.
- 2026Always-OnAgents:A Survey of Persistent Memory, State, and Governance in LLMAgentscited by 5
- 2026Winning by Peeking: Unenforced Budgets and Test-Set Selection Inflate Short-Budget AutoML Comparisonscited by 1
- 2026Inferring Hidden User Models from the Behavior of Personalized LLM Agentscited by 0
- 2026Audience-Bound Persistent Memory: Authorization Across the Memory Lifecyclecited by 0
- 2026Audience-Bound Persistent Memory: Authorization Across the Memory Lifecycle28 Sep 2026
- 2026Inferring Hidden User Models from the Behavior of Personalized LLM Agents3 Sep 2026
- 2026Winning by Peeking: Unenforced Budgets and Test-Set Selection Inflate Short-Budget AutoML Comparisons7 Aug 2026
- 2026Always-OnAgents:A Survey of Persistent Memory, State, and Governance in LLMAgents29 Jun 2026
Similar, not cited
Close in wording, no citation either way
- 2026From Recall to Forgetting: Benchmarking Long-Term Memory for Personalized Agents
- 2026Kernel-Managed Shared Memory for System-Wide Personalization
- 2026MemAudit: Auditing Long-Term Agent Memory via Hidden User-State Recovery
- 2026Controllable Memory Usage: Balancing Anchoring and Innovation in Long-Term Human-Agent Interaction
- 2025Forgetful but Faithful: A Cognitive Memory Architecture and Benchmark for Privacy-Aware Generative Agents
- 2026Structured Distillation for Personalized Agent Memory: 11x Token Reduction with Retrieval Preservation
- 2026Personalize-then-Store: Benchmarking and Learning Personalized Memory for Long-horizon Agents
- 2026Beyond Similarity: Trustworthy Memory Search for Personal AI Agents