mlsanityData to 14 Oct 2026

Deployment-Time Memorization in Foundation-Model Agents

arXiv 2606.10062 ↗First posted 8 Jun 2026cs.AI cs.MAcited by 4

On the map: LLM agents and tool use → Agent memory systems

Abstract (arXiv)

Foundation-model agents are increasingly long-lived systems that remember users across interactions, making memorization an explicit deployment-time function rather than solely a property of model weights. Existing work addresses parametric memorization or audits fixed memory configurations, but does not characterize how memory-design choices jointly shape personalization utility, extraction risk, and deletion fidelity. We study this surface as deployment-time memorization, formulating agent memory as a privacy-utility frontier measured by Personalization Recall (PR) and Adversarial Extraction Rate (AER), and sweeping three memory-design knobs: summarization aggressiveness, retrieval breadth (k), and deletion mode. We further introduce the Forgetting Residue Score (FRS) to quantify whether deleted information remains recoverable from derived memory tiers. On LongMemEval, key-fact summarization reduces canary extraction by 76% on Gemma 3 12B and 64% on GPT-4o-mini while preserving nearly all personalization recall; critically, once content is compressed away, increasing k no longer restores leakage. The same compression, however, induces a deletion-fidelity failure: raw-only deletion leaves derived summary copies recoverable in approximately 20% of instances, and only full-pipeline purge or tombstone redaction drives worst-tier residue to zero. Together, these results establish that persistent agent memory must be evaluated as a first-class memorization mechanism -- assessed by what it helps agents recall, what it makes extractable, and what it can truly erase.

Key facts. Cited by 4 papers in the map (data through 14 Oct 2026). Placed in Agent memory systems, one of 16 directions in LLM agents and tool use. Builds on 16 of its 21 references that are in the map. First posted on arXiv 8 Jun 2026.

Builds on

16 of 21 references.

Closest ancestors · 1

Same direction, newest first

  1. 2026Adaptive Memory Admission Control for LLM Agents4 Mar

Other references · 15

  1. 2020Retrieval-Augmented Generation for Knowledge-Intensive NLP Taskscited by 9,297
  2. 2019Sentence-BERT: Sentence Embeddings using Siamese BERT-Networkscited by 9,170
  3. 2023Generative Agents: Interactive Simulacra of Human Behaviorcited by 3,568
  4. 2016Deep Learning with Differential Privacycited by 2,728
  5. 2020Extracting Training Data from Large Language Modelscited by 2,189
  6. 2023Not what you've signed up for: Compromising Real-World LLM-Integrated Applications with Indirect Prompt Injectioncited by 1,443
  7. 2019Machine Unlearningcited by 1,061
  8. 2018The Secret Sharer: Evaluating and Testing Unintended Memorization in Neural Networkscited by 864
  9. 2022Quantifying Memorization Across Neural Language Modelscited by 763
  10. 2024LongMemEval: Benchmarking Chat Assistants on Long-Term Interactive Memorycited by 707
  11. 2023Scalable Extraction of Training Data from (Production) Language Modelscited by 434
  12. 2023Can LLMs Keep a Secret? Testing Privacy Implications of Language Models via Contextual Integrity Theorycited by 230
3 more
  1. 2025Unveiling Privacy Risks in LLM Agent Memorycited by 101
  2. 2026A Survey on Long-Term Memory Security in LLM Agents: Attacks, Defenses, and Governance Across the Memory Lifecyclecited by 27
  3. 2026Trojan Hippo: Weaponizing Agent Memory for Data Exfiltrationcited by 12

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2026: 4.

4 papers.

Similar, not cited

Close in wording, no citation either way

  1. 2026From Recall to Forgetting: Benchmarking Long-Term Memory for Personalized Agents
  2. 2026Kernel-Managed Shared Memory for System-Wide Personalization
  3. 2026MemAudit: Auditing Long-Term Agent Memory via Hidden User-State Recovery
  4. 2026Controllable Memory Usage: Balancing Anchoring and Innovation in Long-Term Human-Agent Interaction
  5. 2025Forgetful but Faithful: A Cognitive Memory Architecture and Benchmark for Privacy-Aware Generative Agents
  6. 2026Structured Distillation for Personalized Agent Memory: 11x Token Reduction with Retrieval Preservation
  7. 2026Personalize-then-Store: Benchmarking and Learning Personalized Memory for Long-horizon Agents
  8. 2026Beyond Similarity: Trustworthy Memory Search for Personal AI Agents