# RecUserSim: A Realistic and Diverse User Simulator for Evaluating Conversational Recommender Systems

arXiv: https://arxiv.org/abs/2507.22897 · HTML: https://mlsanity.com/paper/2507.22897/

## Key facts

- Cited by 12 papers in the map (data through 14 Oct 2026).
- Builds on 25 of its 41 references that are in the map.
- First posted on arXiv 25 Jun 2025.

## Builds on (most cited first, 25 of 25)

- [Judging LLM-as-a-Judge with MT-Bench and Chatbot Arena](https://mlsanity.com/paper/2306.05685/) (2023)
- [A Survey on Large Language Model based Autonomous Agents](https://mlsanity.com/paper/2308.11432/) (2023)
- [Wide & Deep Learning for Recommender Systems](https://mlsanity.com/paper/1606.07792/) (2016)
- [A Survey on the Memory Mechanism of Large Language Model based Agents](https://mlsanity.com/paper/2404.13501/) (2024)
- [Generative Verifiers: Reward Modeling as Next-Token Prediction](https://mlsanity.com/paper/2408.15240/) (2024)
- [Chat-REC: Towards Interactive and Explainable LLMs-Augmented Recommender System](https://mlsanity.com/paper/2303.14524/) (2023)
- [Bias and Debias in Recommender System: A Survey and Future Directions](https://mlsanity.com/paper/2010.03240/) (2020)
- [From Persona to Personalization: A Survey on Role-Playing Language Agents](https://mlsanity.com/paper/2404.18231/) (2024)
- [Uncovering ChatGPT's Capabilities in Recommender Systems](https://mlsanity.com/paper/2305.02182/) (2023)
- [A Survey on Conversational Recommender Systems](https://mlsanity.com/paper/2004.00646/) (2020)
- [Improving Conversational Recommender Systems via Knowledge Graph based Semantic Fusion](https://mlsanity.com/paper/2007.04032/) (2020)
- [Advances and Challenges in Conversational Recommender Systems: A Survey](https://mlsanity.com/paper/2101.09459/) (2021)
- [Conversational Recommender System](https://mlsanity.com/paper/1806.03277/) (2018)
- [Estimation-Action-Reflection: Towards Deep Interaction Between Conversational and Recommender Systems](https://mlsanity.com/paper/2002.09102/) (2020)
- [Towards Unified Conversational Recommender Systems via Knowledge-Enhanced Prompt Learning](https://mlsanity.com/paper/2206.09363/) (2022)
- [Large Language Models for Generative Recommendation: A Survey and Visionary Discussions](https://mlsanity.com/paper/2309.01157/) (2023)
- [Interactive Path Reasoning on Graph for Conversational Recommendation](https://mlsanity.com/paper/2007.00194/) (2020)
- [Rethinking the Evaluation for Conversational Recommendation in the Era of Large Language Models](https://mlsanity.com/paper/2305.13112/) (2023)
- [Evaluating Conversational Recommender Systems via User Simulation](https://mlsanity.com/paper/2006.08732/) (2020)
- [A Multi-Agent Conversational Recommender System](https://mlsanity.com/paper/2402.01135/) (2024)
- [Towards Question-based Recommender Systems](https://mlsanity.com/paper/2005.14255/) (2020)
- [MemSim: A Bayesian Simulator for Evaluating Memory of LLM-based Personal Assistants](https://mlsanity.com/paper/2409.20163/) (2024)
- [Deep Conversational Recommender in Travel](https://mlsanity.com/paper/1907.00710/) (2019)
- [Pearl: A Review-driven Persona-Knowledge Grounded Conversational Recommendation Dataset](https://mlsanity.com/paper/2403.04460/) (2024)
- [Conversational Recommender System and Large Language Model Are Made for Each Other in E-commerce Pre-sales Dialogue](https://mlsanity.com/paper/2310.14626/) (2023)

## Built on by (most cited first, 12 of 12)

- [RecoWorld: Building Simulated Environments for Agentic Recommender Systems](https://mlsanity.com/paper/2509.10397/) (2025)
- [Agentic Recommender System with Hierarchical Belief-State Memory](https://mlsanity.com/paper/2605.14401/) (2026)
- [Autonomous Information Seeking: A Roadmap for Agentic Recommender Systems](https://mlsanity.com/paper/2607.04433/) (2026)
- [τ-Rec: A Verifiable Benchmark for Agentic Recommender Systems](https://mlsanity.com/paper/2606.10156/) (2026)
- [CORAL: An LLM-Native Harness for Production Recommender Systems](https://mlsanity.com/paper/2609.02730/) (2026)
- [Who Does What? Archetypes of Roles Assigned to LLMs During Human-AI Decision-Making](https://mlsanity.com/paper/2602.11924/) (2026)
- [Benchmarked Yet Not Measured -- Generative AI Should be Evaluated Against Real-World Utility](https://mlsanity.com/paper/2605.06856/) (2026)
- [Prompt Optimization for User Simulation in Conversational Recommender Systems: A Multi-Objective Framework](https://mlsanity.com/paper/2607.00010/) (2026)
- [SR-Agent: An Experience-Driven Agentic Framework for Post-Ranking Strategy Refinement in E-Commerce Recommendation](https://mlsanity.com/paper/2607.17719/) (2026)
- [SimTrace: Grounded Multimodal User Trajectories Generation for Online User Modeling](https://mlsanity.com/paper/2609.38397/) (2026)
- [Back in Style: A Sociolinguistic Approach to Authoring and Measuring Persona Fidelity in User Simulation](https://mlsanity.com/paper/2610.10988/) (2026)
- [PsyBer-Agent: A Psychology-Driven User Agent for Recommender Systems](https://mlsanity.com/paper/2610.13989/) (2026)

## Abstract (arXiv)

Conversational recommender systems (CRS) enhance user experience through multi-turn interactions, yet evaluating CRS remains challenging. User simulators can provide comprehensive evaluations through interactions with CRS, but building realistic and diverse simulators is difficult. While recent work leverages large language models (LLMs) to simulate user interactions, they still fall short in emulating individual real users across diverse scenarios and lack explicit rating mechanisms for quantitative evaluation. To address these gaps, we propose RecUserSim, an LLM agent-based user simulator with enhanced simulation realism and diversity while providing explicit scores. RecUserSim features several key modules: a profile module for defining realistic and diverse user personas, a memory module for tracking interaction history and discovering unknown preferences, and a core action module inspired by Bounded Rationality theory that enables nuanced decision-making while generating more fine-grained actions and personalized responses. To further enhance output control, a refinement module is designed to fine-tune final responses. Experiments demonstrate that RecUserSim generates diverse, controllable outputs and produces realistic, high-quality dialogues, even with smaller base LLMs. The ratings generated by RecUserSim show high consistency across different base LLMs, highlighting its effectiveness for CRS evaluation.