# Mem-T: Densifying Rewards for Long-Horizon Memory Agents

arXiv: https://arxiv.org/abs/2601.23014 · HTML: https://mlsanity.com/paper/2601.23014/

## Key facts

- Cited by 18 papers in the map (data through 14 Oct 2026).
- Builds on 39 of its 45 references that are in the map.
- First posted on arXiv 30 Jan 2026.

## Builds on (most cited first, 30 of 39)

- [DeepSeekMath: Pushing the Limits of Mathematical Reasoning in Open Language Models](https://mlsanity.com/paper/2402.03300/) (2024)
- [Retrieval-Augmented Generation for Knowledge-Intensive NLP Tasks](https://mlsanity.com/paper/2005.11401/) (2020)
- [HotpotQA: A Dataset for Diverse, Explainable Multi-hop Question Answering](https://mlsanity.com/paper/1809.09600/) (2018)
- [AutoGen: Enabling Next-Gen LLM Applications via Multi-Agent Conversation](https://mlsanity.com/paper/2308.08155/) (2023)
- [MetaGPT: Meta Programming for A Multi-Agent Collaborative Framework](https://mlsanity.com/paper/2308.00352/) (2023)
- [MemGPT: Towards LLMs as Operating Systems](https://mlsanity.com/paper/2310.08560/) (2023)
- [M3-Embedding: Multi-Linguality, Multi-Functionality, Multi-Granularity Text Embeddings Through Self-Knowledge Distillation](https://mlsanity.com/paper/2402.03216/) (2024)
- [A-MEM: Agentic Memory for LLM Agents](https://mlsanity.com/paper/2502.12110/) (2025)
- [ExpeL: LLM Agents Are Experiential Learners](https://mlsanity.com/paper/2308.10144/) (2023)
- [Evaluating Very Long-Term Conversational Memory of LLM Agents](https://mlsanity.com/paper/2402.17753/) (2024)
- [Mem0: Building Production-Ready AI Agents with Scalable Long-Term Memory](https://mlsanity.com/paper/2504.19413/) (2025)
- [LongMemEval: Benchmarking Chat Assistants on Long-Term Interactive Memory](https://mlsanity.com/paper/2410.10813/) (2024)
- [The NarrativeQA Reading Comprehension Challenge](https://mlsanity.com/paper/1712.07040/) (2017)
- [MemoryBank: Enhancing Large Language Models with Long-Term Memory](https://mlsanity.com/paper/2305.10250/) (2023)
- [Zep: A Temporal Knowledge Graph Architecture for Agent Memory](https://mlsanity.com/paper/2501.13956/) (2025)
- [ToolRL: Reward is All Tool Learning Needs](https://mlsanity.com/paper/2504.13958/) (2025)
- [Memory in the Age of AI Agents](https://mlsanity.com/paper/2512.13564/) (2025)
- [Long-context LLMs Struggle with Long In-context Learning](https://mlsanity.com/paper/2404.02060/) (2024)
- [MemAgent: Reshaping Long-Context LLM with Multi-Conv RL-based Memory Agent](https://mlsanity.com/paper/2507.02259/) (2025)
- [MEM1: Learning to Synergize Memory and Reasoning for Efficient Long-Horizon Agents](https://mlsanity.com/paper/2506.15841/) (2025)
- [Memory-R1: Enhancing Large Language Model Agents to Manage and Utilize Memories via Reinforcement Learning](https://mlsanity.com/paper/2508.19828/) (2025)
- [LightMem: Lightweight and Efficient Memory-Augmented Generation](https://mlsanity.com/paper/2510.18866/) (2025)
- [Scaling Long-Horizon LLM Agent via Context-Folding](https://mlsanity.com/paper/2510.11967/) (2025)
- [ReSum: Unlocking Long-Horizon Search Intelligence via Context Summarization](https://mlsanity.com/paper/2509.13313/) (2025)
- [MemRL: Self-Evolving Agents via Runtime Reinforcement Learning on Episodic Memory](https://mlsanity.com/paper/2601.03192/) (2026)
- [AgentFold: Long-Horizon Web Agents with Proactive Context Management](https://mlsanity.com/paper/2510.24699/) (2025)
- [How Memory Management Impacts LLM Agents: An Empirical Study of Experience-Following Behavior](https://mlsanity.com/paper/2505.16067/) (2025)
- [MemEvolve: Meta-Evolution of Agent Memory Systems](https://mlsanity.com/paper/2512.18746/) (2025)
- [MemBench: Towards More Comprehensive Evaluation on the Memory of LLM-based Agents](https://mlsanity.com/paper/2506.21605/) (2025)
- [Tree Search for LLM Agent Reinforcement Learning](https://mlsanity.com/paper/2509.21240/) (2025)

## Built on by (most cited first, 18 of 18)

- [Learning to Remember: End-to-End Training of Memory Agents for Long-Context Reasoning](https://mlsanity.com/paper/2602.18493/) (2026)
- [MemAudit: Auditing Long-Term Agent Memory via Hidden User-State Recovery](https://mlsanity.com/paper/2606.24595/) (2026)
- [TRUSTMEM: Learning Trustworthy Memory Consolidation for LLM Agents with Long-Term Memory](https://mlsanity.com/paper/2606.25161/) (2026)
- [MemQ: Integrating Q-Learning into Self-Evolving Memory Agents over Provenance DAGs](https://mlsanity.com/paper/2605.08374/) (2026)
- [AttriMem: Attribution-Guided Process Feedback for Agent Memory Construction](https://mlsanity.com/paper/2607.21106/) (2026)
- [LazyMem: Retrieve Broadly, Construct Selectively for Efficient Long-Term Agent Memory](https://mlsanity.com/paper/2607.22690/) (2026)
- [Learning What to Remember: Observability-Safe Memory Retention via Constrained Optimization for Long-Horizon Language Agents](https://mlsanity.com/paper/2606.10616/) (2026)
- [MemTrace: Probing What Final Accuracy Misses in Long-Term Memory](https://mlsanity.com/paper/2606.17328/) (2026)
- [Agent-ValueBench: A Comprehensive Benchmark for Evaluating Agent Values](https://mlsanity.com/paper/2605.10365/) (2026)
- [MedMemoryBench: Benchmarking Agent Memory in Personalized Healthcare](https://mlsanity.com/paper/2605.11814/) (2026)
- [PersonaTree: Structured Lifecycle Memory for Person Understanding in LLM Agents](https://mlsanity.com/paper/2606.04780/) (2026)
- [MMLA: Memory-Mediated Learning Architecture for Predictive Dual-State Adaptation](https://mlsanity.com/paper/2606.28876/) (2026)
- [Dual-Grained Agent Memory and Shapley Context Attribution for Multimodal Agentic Learner](https://mlsanity.com/paper/2608.23268/) (2026)
- [Back to Basics: Let Conversational Agents Remember with Just Retrieval and Generation](https://mlsanity.com/paper/2604.11628/) (2026)
- [Beyond Dyadic Memory: Interaction-Aware Multimodal Memory with Adaptive Agentic Retrieval for Multi-Party Spoken Conversations](https://mlsanity.com/paper/2609.32522/) (2026)
- [Just-In-Time Agent Memory with Runtime Agentic Research](https://mlsanity.com/paper/2609.34385/) (2026)
- [Coding Agent Memory Post-training: Unlocking the Memory Potential of Pre-trained File Operations for Long-Horizon Tasks via Reinforcement Learning](https://mlsanity.com/paper/2609.34422/) (2026)
- [MemPilot: Orchestrating On-Demand Multimodal Memory Curation for LLM Agents](https://mlsanity.com/paper/2610.06830/) (2026)

## Abstract (arXiv)

Memory agents, which depart from predefined memory-processing pipelines by endogenously managing the processing, storage, and retrieval of memories, have garnered increasing attention for their autonomy and adaptability. However, existing training paradigms remain constrained: agents often traverse long-horizon sequences of memory operations before receiving sparse and delayed rewards, which hinders truly end-to-end optimization of memory management policies. To address this limitation, we introduce Mem-T, an autonomous memory agent that interfaces with a lightweight hierarchical memory database to perform dynamic updates and multi-turn retrieval over streaming inputs. To effectively train long-horizon memory management capabilities, we further propose MoT-GRPO, a tree-guided reinforcement learning framework that transforms sparse terminal feedback into dense, step-wise supervision via memory operation tree backpropagation and hindsight credit assignment, thereby enabling the joint optimization of memory construction and retrieval. Extensive experiments demonstrate that Mem-T is (1) high-performing, surpassing frameworks such as A-Mem and Mem0 by up to 14.92%, and (2) economical, operating on a favorable accuracy-efficiency Pareto frontier and reducing inference tokens per query by ~24.45% relative to GAM without sacrificing performance.