MemoryRewardBench: Benchmarking Reward Models for Long-Term Memory Management in Large Language Models
On the map: LLM alignment / RLHF / preference optimisation → Reward modeling and RLHF
Abstract (arXiv)
Existing works increasingly adopt memory-centric mechanisms to process long contexts in a segment manner, and effective memory management is one of the key capabilities that enables large language models to effectively propagate information across the entire sequence. Therefore, leveraging reward models (RMs) to automatically and reliably evaluate memory quality is critical. In this work, we introduce MemoryRewardBench, the first benchmark to systematically study the ability of RMs to evaluate long-term memory management processes. MemoryRewardBench covers both long-context comprehension and long-form generation tasks, featuring 10 distinct settings with different memory management patterns, with context length ranging from 8K to 128K tokens. Evaluations on 13 cutting-edge RMs indicate a diminishing performance gap between open-source and proprietary models, with newer-generation models consistently outperforming their predecessors regardless of parameter count. We further expose the capabilities and fundamental limitations of current RMs in evaluating LLM memory management across diverse settings.
Key facts. Cited by 2 papers in the map (data through 14 Oct 2026). Placed in Reward modeling and RLHF, one of 10 directions in LLM alignment / RLHF / preference optimisation. Builds on 39 of its 53 references that are in the map. First posted on arXiv 17 Jan 2026.
Builds on
39 of 53 references.
Closest ancestors · 5
Same direction, newest first
- 2025LongRM: Revealing and Unlocking the Context Boundary of Reward Modeling8 Oct
- 2025Pre-Trained Policy Discriminators are General Reward Models7 Jul
- 2025Reward Models in Deep Reinforcement Learning: A Survey18 Jun
- 2024Skywork-Reward: Bag of Tricks for Reward Modeling in LLMs24 Oct
- 2023Reinforcement Learning with Human Feedback: Learning Dynamic Choices via Pessimism29 May
Other references · 34
- 2023Judging LLM-as-a-Judge with MT-Bench and Chatbot Arenacited by 8,626
- 2023LongBench: A Bilingual, Multitask Benchmark for Long Context Understandingcited by 1,417
- 2025A-MEM: Agentic Memory for LLM Agentscited by 1,063
- 2024Evaluating Very Long-Term Conversational Memory of LLM Agentscited by 830
- 2023AlpacaFarm: A Simulation Framework for Methods that Learn from Human Feedbackcited by 748
- 2024LongMemEval: Benchmarking Chat Assistants on Long-Term Interactive Memorycited by 707
- 2025GLM-4.5: Agentic, Reasoning, and Coding (ARC) Foundation Modelscited by 467
- 2024LongBench v2: Towards Deeper Understanding and Reasoning on Realistic Long-context Multitaskscited by 379
- 2025Evaluating Memory in LLM Agents via Incremental Multi-Turn Interactionscited by 254
- 2025MemAgent: Reshaping Long-Context LLM with Multi-Conv RL-based Memory Agentcited by 246
- 2024From r to Q^*: Your Language Model is Secretly a Q-Functioncited by 242
- 2024BABILong: Testing the Limits of LLMs with Long Context Reasoning-in-a-Haystackcited by 215
22 more
- 2024Free Process Rewards without Process Labelscited by 165
- 2025Do LLMs Recognize Your Preferences? Evaluating Personalized Preference Following in LLMscited by 148
- 2025Scaling Long-Horizon LLM Agent via Context-Foldingcited by 148
- 2025On Memory Construction and Retrieval for Personalized Conversational Agentscited by 105
- 2025AgentFold: Long-Horizon Web Agents with Proactive Context Managementcited by 103
- 2025PersonaMem-v2: Towards Personalized Intelligence via Learning Implicit User Personas and Agentic Memorycited by 88
- 2025MemBench: Towards More Comprehensive Evaluation on the Memory of LLM-based Agentscited by 84
- 2025MemoryBench: A Benchmark for Memory and Continual Learning in LLM Systemscited by 61
- 2025Beyond a Million Tokens: Benchmarking and Enhancing Long-Term Memory in LLMscited by 60
- 2024StreamBench: Towards Benchmarking Continuous Improvement of Language Agentscited by 47
- 2024LongGenBench: Benchmarking Long-Form Generation in Long Context LLMscited by 47
- 2025Building Self-Evolving Agents via Experience-Driven Lifelong Learning: A Framework and Benchmarkcited by 45
- 2024PerLTQA: A Personal Long-Term Memory Dataset for Memory Classification, Retrieval, and Synthesis in Question Answeringcited by 34
- 2025LongProc: Benchmarking Long-Context Language Models on Long Procedural Generationcited by 29
- 2025StoryBench: A Dynamic Benchmark for Evaluating Long-Term Memory with Multi Turnscited by 20
- 2025LoCoBench-Agent: An Interactive Benchmark for LLM Agents in Long-Context Software Engineeringcited by 16
- 2025LongEval: A Comprehensive Analysis of Long-Text Generation Through a Plan-based Paradigmcited by 14
- 2025MEMTRACK: Evaluating Long-Term Memory and State Tracking in Multi-Platform Dynamic Agent Environmentscited by 13
- 2024Stable Hadamard Memory: Revitalizing Memory-Augmented Agents for Reinforcement Learningcited by 9
- 2025Minerva: A Programmable Memory Test Benchmark for Language Modelscited by 9
- 2025Think Twice: Branch-and-Rethink Reasoning Reward Modelcited by 7
- 2025LOOM-Scope: a comprehensive and efficient LOng-cOntext Model evaluation frameworkcited by 5
Built on by
Citing papers per year: 2026: 2.
2 papers.
Similar, not cited
Close in wording, no citation either way
- 2026Is One Score Enough? Rethinking the Evaluation of Sequentially Evolving LLM Memory
- 2026MemSifter: Offloading LLM Memory Retrieval via Outcome-Driven Proxy Reasoning
- 2025ToolRM: Outcome Reward Models for Tool-Calling Large Language Models
- 2025Rethinking Reward Models for Multi-Domain Test-Time Scaling
- 2026Fine-Mem: Fine-Grained Feedback Alignment for Long-Horizon Memory Management
- 2024RMB: Comprehensively Benchmarking Reward Models in LLM Alignment
- 2025Memory in Large Language Models: Mechanisms, Evaluation and Evolution
- 2026IF-RewardBench: Benchmarking Judge Models for Instruction-Following Evaluation