Scaling Self-Evolving Agents via Parametric Memory

Existing memory-augmented LLM agents store past experience exclusively in prompt space, as textual summaries or retrieved passages, while keeping model parameters frozen throughout a rollout. Such agents can look up what they have seen but cannot learn from it: their policy is unchanged by experience, and any information dropped from the context is permanently lost. We introduce textttTMEM, a self-evolving parametric memory framework in which the agent not only compresses history into explicit memory but also absorbs distilled supervision into fast LoRA weights Deltat via lightweight online updates, genuinely altering its future behavior within a single episode. We formalize this as an agentic decision process with fast-weight rollout dynamics: actions are sampled from πθ0+Deltat, while extraction actions produce supervision that updates Deltat for subsequent decisions. This view makes the extraction policy directly optimizable by RL: training θ0 improves not only task actions but also the quality of the data used for online LoRA adaptation. We further propose SVD-based initialization of the LoRA subspace to accelerate online convergence. Experiments on LoCoMo, LongMemEval-S, multi-objective search, and CL-Bench show that textttTMEM consistently outperforms summary-based and retrieval-based baselines across different model scales.