# RecRM-Bench: Benchmarking Multidimensional Reward Modeling for Agentic Recommender Systems

arXiv: https://arxiv.org/abs/2605.11874 · HTML: https://mlsanity.com/paper/2605.11874/

## Key facts

- Cited by 4 papers in the map (data through 14 Oct 2026).
- Placed in Agentic recommender systems, one of 16 directions in LLM agents and tool use.
- Builds on 19 of its 71 references that are in the map.
- First posted on arXiv 12 May 2026.

## Placed in

- LLM agents and tool use / [Agentic recommender systems](https://mlsanity.com/agents/agentic-recommender-systems/)

## Builds on (most cited first, 19 of 19)

- [Self-Attentive Sequential Recommendation](https://mlsanity.com/paper/1808.09781/) (2018)
- [Image-based Recommendations on Styles and Substitutes](https://mlsanity.com/paper/1506.04757/) (2015)
- [LLMRec: Large Language Models with Graph Augmentation for Recommendation](https://mlsanity.com/paper/2311.00423/) (2023)
- [RecMind: Large Language Model Powered Agent For Recommendation](https://mlsanity.com/paper/2308.14296/) (2023)
- [Process vs. Outcome Reward: Which is Better for Agentic RAG Reinforcement Learning](https://mlsanity.com/paper/2505.14069/) (2025)
- [A Survey on LLM-powered Agents for Recommender Systems](https://mlsanity.com/paper/2502.10050/) (2025)
- [AGENTIF: Benchmarking Instruction Following of Large Language Models in Agentic Scenarios](https://mlsanity.com/paper/2505.16944/) (2025)
- [RAH! RecSys-Assistant-Human: A Human-Centered Recommendation Framework with LLM Agents](https://mlsanity.com/paper/2308.09904/) (2023)
- [AgentRecBench: Benchmarking LLM Agent-based Personalized Recommender Systems](https://mlsanity.com/paper/2505.19623/) (2025)
- [Towards Next-Generation Recommender Systems: A Benchmark for Personalized Recommendation Assistant with LLMs](https://mlsanity.com/paper/2503.09382/) (2025)
- [RecoWorld: Building Simulated Environments for Agentic Recommender Systems](https://mlsanity.com/paper/2509.10397/) (2025)
- [Thought-Augmented Planning for LLM-Powered Interactive Recommender Agent](https://mlsanity.com/paper/2506.23485/) (2025)
- [Reindex-Then-Adapt: Improving Large Language Models for Conversational Recommendation](https://mlsanity.com/paper/2405.12119/) (2024)
- [MR.Rec: Synergizing Memory and Reasoning for Personalized Recommendation Assistant with LLMs](https://mlsanity.com/paper/2510.14629/) (2025)
- [JDsearch: A Personalized Product Search Dataset with Real Queries and Full Interactions](https://mlsanity.com/paper/2305.14810/) (2023)
- [AgenticShop: Benchmarking Agentic Product Curation for Personalized Web Shopping](https://mlsanity.com/paper/2602.12315/) (2026)
- [Beyond Single Labels: Improving Conversational Recommendation through LLM-Powered Data Augmentation](https://mlsanity.com/paper/2508.05657/) (2025)
- [DARLR: Dual-Agent Offline Reinforcement Learning for Recommender Systems with Dynamic Reward](https://mlsanity.com/paper/2505.07257/) (2025)
- [WeMusic-Agent: Efficient Conversational Music Recommendation via Knowledge Internalization and Agentic Boundary Learning](https://mlsanity.com/paper/2512.16108/) (2025)

## Built on by (most cited first, 4 of 4)

- [CheckerBench: Can Long-Horizon Agents Synthesize Static-Analysis Checkers?](https://mlsanity.com/paper/2610.07557/) (2026)
- [RAG-Stress: Probing the Limits of Evidence Reliance in Retrieval-Augmented Generation](https://mlsanity.com/paper/2610.11183/) (2026)
- [ATLAS: Dual-Horizon Diagnostic Evaluation for Industrial Tool-Use Agents](https://mlsanity.com/paper/2608.30685/) (2026)
- [WorkRover: Scaling Long-Horizon Cowork Agent Training Via Recursive Environment Evolution](https://mlsanity.com/paper/2610.18067/) (2026)

## Abstract (arXiv)

The integration of Large Language Model (LLM) agents is transforming recommender systems from simple query-item matching towards deeply personalized and interactive recommendations. Reinforcement Learning (RL) provides an essential framework for the optimization of these agents in recommendation tasks. However, current methodologies remain limited by a reliance on single dimensional outcome-based rewards that focus exclusively on final user interactions, overlooking critical intermediate capabilities, such as instruction following and complex intent understanding. Despite the necessity for designing multi-dimensional reward, the field lacks a standardized benchmark to facilitate this development. To bridge this gap, we introduce RecRM-Bench, the largest and most comprehensive benchmark to date for agentic recommender systems. It comprises over 1 million structured entries across four core evaluation dimensions: instruction following, factual consistency, query-item relevance, and fine-grained user behavior prediction. By supporting comprehensive assessment from syntactic compliance to complex intent grounding and preference modeling, RecRM-Bench provides a foundational dataset for training sophisticated reward models. Furthermore, we propose a systematic framework for the construction of multi-dimensional reward models and the integration of a hybrid reward function, establishing a robust foundation for developing reliable and highly capable agentic recommender systems. The complete RecRM-Bench dataset is publicly available at https://huggingface.co/datasets/wwzeng/RecRM-Bench.