Memory Depth, Not Memory Access: Selective Parametric Consolidation for Long-Running Language Agents
On the map: LLM agents and tool use → Agent memory systems
Abstract (arXiv)
Long-running language agents need more than memory access. Retrieval systems can fetch past facts at query time, but they do not decide which experiences should continue to shape behavior after the working context is unloaded. We study this separate problem as memory depth: durable goal-conditioned tendencies written into a small parametric store. We introduce the loop-drift protocol, a controlled stress test in which the retrieval index remains intact while working context is unloaded and goal-conditioned behavior must persist under long-loop interference. We evaluate EVAF, a surprise- and valence-gated LoRA consolidation mechanism. Across GPT-2 and TinyLlama, retrieval is strongest on shallow factual recall (short-fact accuracy 0.956--0.973), while EVAF is strongest on goal persistence and post-unload recovery (0.812--0.904) with only 2--3 parametric writes per 200 events. Mechanism controls show that selective consolidation factorizes into two controllable dimensions: selection and actuation. Matched random gates isolate selection beyond sparse writing; fixed-inner controls across GPT-2, TinyLlama, and Mistral-7B show that inner-loop write strength is model-dependent; and a Mistral-7B matched-gate inversion reveals asymmetric selection-actuation coupling under miscalibrated actuation. Public Memora event streams serve as an external diagnostic, exposing stale-memory invalidation as an unresolved boundary. Within this probe, selective parametric consolidation supplies memory depth distinct from and complementary to retrieval access.
Key facts. Cited by 1 paper in the map (data through 14 Oct 2026). Placed in Agent memory systems, one of 16 directions in LLM agents and tool use. Builds on 5 of its 17 references that are in the map. First posted on arXiv 25 Jun 2026.
Builds on
5 of 17 references.
References · 5
- 2020Retrieval-Augmented Generation for Knowledge-Intensive NLP Taskscited by 9,297
- 2023MemGPT: Towards LLMs as Operating Systemscited by 1,389
- 2018Experience Replay for Continual Learningcited by 1,095
- 2024LongMemEval: Benchmarking Chat Assistants on Long-Term Interactive Memorycited by 707
- 2026Rethinking LoRA Memory Through the Lens of KV Cache Compressioncited by 2
Built on by
Citing papers per year: 2026: 1.
1 paper.
Similar, not cited
Close in wording, no citation either way
- 2026What to Keep, What to Forget: A Rate--Distortion View of Memory Compaction in LLMs and Agents
- 2026Auto-Dreamer: Learning Offline Memory Consolidation for Language Agents
- 2026CAMeR: Keyword-Gated Hybrid Activation for Adaptive Memory Retention in LLM Agents
- 2026Retain or Consolidate? Budget-Dependent Operator Selection for Language Agent Memory
- 2026Dual-Layer Agentic Memory with Fast Write Routing and Slow Consolidation
- 2025Memory in Large Language Models: Mechanisms, Evaluation and Evolution
- 2026Heavy-Tailed Memory Traces in Long-Horizon Language Agents
- 2024On the Structural Memory of LLM Agents