mlsanityData to 14 Oct 2026

CMI-Mem: Toward Generalizable Long-Term Memory Management via CMI-Augmented Reinforcement Learning

arXiv 2607.20553 ↗First posted 15 Jul 2026cs.AI cs.CLcited by 1

On the map: Deep reinforcement learning methods → RL training of language and agent models

Abstract (arXiv)

Memory Manager models are pivotal in agent systems. Existing reinforcement-learning methods commonly use LLM-judged synthetic question-answer (QA) pairs: this provides useful downstream task grounding, but values memory through a sampled query distribution and a fixed reader. We propose CMI-Mem, a lightweight RL memory manager with a hybrid reward. Its extrinsic QA term measures end-task correctness, while its intrinsic Conditional Mutual Information (CMI) term evaluates the information contributed by new conversational inputs relative to the current memory state without conditioning on a sampled QA query. The two signals are complementary: QA anchors task utility, whereas CMI provides per-operation supervision for relevant, non-redundant memory construction. Experiments demonstrate improved transfer across memory-use scenarios, together with more efficient training and inference from the per-operation CMI signal. Our codes are available at: https://github.com/Wyb0627/CMIMem , and the CMI-Mem-4B model checkpoint is available at: https://www.modelscope.cn/models/wyb0627/CMIMem-4B

Key facts. Cited by 1 paper in the map (data through 14 Oct 2026). Placed in RL training of language and agent models, one of 15 directions in Deep reinforcement learning methods. Builds on 25 of its 48 references that are in the map. First posted on arXiv 15 Jul 2026.

Builds on

25 of 48 references.

References · 25

  1. 2024DeepSeekMath: Pushing the Limits of Mathematical Reasoning in Open Language Modelscited by 9,379
  2. 2025Qwen3 Embedding: Advancing Text Embedding and Reranking Through Foundation Modelscited by 1,261
  3. 2024LongMemEval: Benchmarking Chat Assistants on Long-Term Interactive Memorycited by 707
  4. 2023An Empirical Study of Catastrophic Forgetting in Large Language Models During Continual Fine-tuningcited by 639
  5. 2023Fine-Grained Human Feedback Gives Better Rewards for Language Model Trainingcited by 401
  6. 2025Zep: A Temporal Knowledge Graph Architecture for Agent Memorycited by 400
  7. 2025ToolRL: Reward is All Tool Learning Needscited by 362
  8. 2021Beyond Goldfish Memory: Long-Term Open-Domain Conversationcited by 267
  9. 2025Evaluating Memory in LLM Agents via Incremental Multi-Turn Interactionscited by 254
  10. 2024VinePPO: Refining Credit Assignment in RL Training of LLMscited by 183
  11. 2025MIRIX: Multi-Agent Memory System for LLM-Based Agentscited by 173
  12. 2019Synthetic QA Corpora Generation with Roundtrip Consistencycited by 143
13 more
  1. 2025MemOS: A Memory OS for AI Systemcited by 138
  2. 2025Right Question is Already Half the Answer: Fully Unsupervised LLM Reasoning Incentivizationcited by 128
  3. 2024Dynamic Memory Compression: Retrofitting LLMs for Accelerated Inferencecited by 116
  4. 2024AriGraph: Learning Knowledge Graph World Models with Episodic Memory for LLM Agentscited by 102
  5. 2025Overcoming catastrophic forgetting in neural networkscited by 55
  6. 2026MemBuilder: Reinforcing LLMs for Long-Term Memory Construction via Attributed Dense Rewardscited by 13
  7. 2025AssoMem: Scalable Memory QA with Multi-Signal Associative Retrievalcited by 11
  8. 2025Memory-QA: Answering Recall Questions Based on Multimodal Memoriescited by 10
  9. 2025ELIP: Enhanced Visual-Language Foundation Models for Image Retrievalcited by 9
  10. 2026Fine-Mem: Fine-Grained Feedback Alignment for Long-Horizon Memory Managementcited by 8
  11. 2025AGRAG: Advanced Graph-based Retrieval-Augmented Generation for LLMscited by 7
  12. 2021Regularization and Reparameterization Avoid Vanishing Gradients in Sigmoid-Type Networkscited by 5
  13. 2025Alleviating Distribution Shift in Synthetic Data for Machine Translation Quality Estimationcited by 4

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2026: 1.

1 paper.

  1. 2026MemCodex: Self-Programming Hierarchical Memory for Language Agentscited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2025Mem-α: Learning Memory Construction via Reinforcement Learning
  2. 2025Memory-R1: Enhancing Large Language Model Agents to Manage and Utilize Memories via Reinforcement Learning
  3. 2026Memory as a Controlled Process: Learned Adaptive Memory Management for LLM Agents
  4. 2025MEM1: Learning to Synergize Memory and Reasoning for Efficient Long-Horizon Agents
  5. 2026Agentic Memory: Learning Unified Long-Term and Short-Term Memory Management for Large Language Model Agents
  6. 2026MemGym: a Long-Horizon Memory Environment for LLM Agents
  7. 2025MemSearcher: Training LLMs to Reason, Search and Manage Memory via End-to-End Reinforcement Learning
  8. 2026MemCo: Memory-Centric Collaboration for Generalizing LLM Agents to Unseen Environments