Learning What to Remember: A Cognitively Grounded Multi-Factor Value Model for Agentic Memory
On the map: LLM agents and tool use → Agent memory systems
Abstract (arXiv)
Long-running LLM agents accumulate interaction histories far larger than any context window, forcing a standing decision: what to encode deeply, what to forget, and what to retrieve under a fixed memory budget. Production systems answer with semantic similarity or recency -- both mis-specified for the forgetting decision, which is made at consolidation time before the future query is known. We propose a multi-factor memory value function V(m)=sumi wi fi(m) over seven interpretable factors (emotional intensity, goal relevance, value alignment, self/user relevance, task utility, reliability, and usage history) drawn from cognitive psychology, whose weights are learned from a downstream objective by a gradient-free optimiser, and whose single scalar uniformly controls encoding depth, forget risk, and retrieval rank. We make a methodological point: on LongMemEval, scoring goal relevance against the held-out evaluation question saturates gold-evidence retention at ≈ 0.98 -- this measures retrieval, not forgetting. In the realistic blind regime, a learned multi-factor value retains 0.770 ± 0.011 of gold evidence across 479 usable cases, versus 0.657 for uniform weights, 0.518 for the best single factor, and 0.368 for recency; every paired gap's 95% bootstrap CI is above zero, and a neural network over the same factors ties the linear model. The learned weights are interpretable -- reliability, emotional intensity, and self/user relevance dominate, while query-time goal similarity is correctly down-weighted for the forgetting decision. A controlled synthetic task with planted confounds confirms the learner recovers a separating weighting (1.00 retention) where uniform weighting fails (0.62). The substrate is open-source; all experiments run on a single CPU with no API calls.
Key facts. No citing papers in the map yet (data through 7 Oct 2026). Placed in Agent memory systems, one of 16 directions in LLM agents and tool use. Builds on 9 of its 21 references that are in the map. First posted on arXiv 11 Jun 2026.
Builds on
9 of 21 references.
References · 9
- 2020Retrieval-Augmented Generation for Knowledge-Intensive NLP Taskscited by 9,160
- 2019Sentence-BERT: Sentence Embeddings using Siamese BERT-Networkscited by 9,049
- 2023Generative Agents: Interactive Simulacra of Human Behaviorcited by 3,500
- 2023MemGPT: Towards LLMs as Operating Systemscited by 1,295
- 2024LongMemEval: Benchmarking Chat Assistants on Long-Term Interactive Memorycited by 656
- 2023MemoryBank: Enhancing Large Language Models with Long-Term Memorycited by 539
- 2023Cognitive Architectures for Language Agentscited by 422
- 2016The CMA Evolution Strategy: A Tutorialcited by 364
- 2025MemOS: A Memory OS for AI Systemcited by 129
Built on by
Not cited yet.
Similar, not cited
Close in wording, no citation either way
- 2026RaMem: Contextual Reinstatement for Long-term Agentic Memory
- 2026From Recall to Forgetting: Benchmarking Long-Term Memory for Personalized Agents
- 2026What to Keep, What to Forget: A Rate--Distortion View of Memory Compaction in LLMs and Agents
- 2026Memory Control Signals Emerge Before Action in Long Horizon Agents
- 2026When Does Memory Help? A Cost-Aware Evaluation of Long-Term Memory in Tool-Using LLM Agents
- 2026The Epistemics of Agent Memory: Measuring, and Governing, the Consolidation Decision in Long-Horizon LLM Agents
- 2025Forgetting as a Feature: Cognitive Alignment of Large Language Models
- 2026Memory as a Controlled Process: Learned Adaptive Memory Management for LLM Agents