# How Reliable is Your Simulator? Analysis on the Limitations of Current LLM-based User Simulators for Conversational Recommendation

arXiv: https://arxiv.org/abs/2403.16416 · HTML: https://mlsanity.com/paper/2403.16416/

## Key facts

- Cited by 27 papers in the map (data through 14 Oct 2026).
- Builds on 25 of its 31 references that are in the map.
- First posted on arXiv 25 Mar 2024.

## Builds on (most cited first, 25 of 25)

- [BART: Denoising Sequence-to-Sequence Pre-training for Natural Language Generation, Translation, and Comprehension](https://mlsanity.com/paper/1910.13461/) (2019)
- [A Survey of Large Language Models](https://mlsanity.com/paper/2303.18223/) (2023)
- [The Rise and Potential of Large Language Model Based Agents: A Survey](https://mlsanity.com/paper/2309.07864/) (2023)
- [Neural Collaborative Filtering](https://mlsanity.com/paper/1708.05031/) (2017)
- [TALLRec: An Effective and Efficient Tuning Framework to Align Large Language Model with Recommendation](https://mlsanity.com/paper/2305.00447/) (2023)
- [Large Language Models are Zero-Shot Rankers for Recommender Systems](https://mlsanity.com/paper/2305.08845/) (2023)
- [Chat-REC: Towards Interactive and Explainable LLMs-Augmented Recommender System](https://mlsanity.com/paper/2303.14524/) (2023)
- [Towards Deep Conversational Recommendations](https://mlsanity.com/paper/1812.07617/) (2018)
- [Uncovering ChatGPT's Capabilities in Recommender Systems](https://mlsanity.com/paper/2305.02182/) (2023)
- [Recommender AI Agent: Integrating Large Language Models for Interactive Recommendations](https://mlsanity.com/paper/2308.16505/) (2023)
- [Large Language Models as Zero-Shot Conversational Recommenders](https://mlsanity.com/paper/2308.10053/) (2023)
- [Improving Conversational Recommender Systems via Knowledge Graph based Semantic Fusion](https://mlsanity.com/paper/2007.04032/) (2020)
- [Advances and Challenges in Conversational Recommender Systems: A Survey](https://mlsanity.com/paper/2101.09459/) (2021)
- [Conversational Recommender System](https://mlsanity.com/paper/1806.03277/) (2018)
- [Towards Knowledge-Based Recommender Dialog System](https://mlsanity.com/paper/1908.05391/) (2019)
- [Estimation-Action-Reflection: Towards Deep Interaction Between Conversational and Recommender Systems](https://mlsanity.com/paper/2002.09102/) (2020)
- [On Generative Agents in Recommendation](https://mlsanity.com/paper/2310.10108/) (2023)
- [Towards Unified Conversational Recommender Systems via Knowledge-Enhanced Prompt Learning](https://mlsanity.com/paper/2206.09363/) (2022)
- [Testing the General Deductive Reasoning Capacity of Large Language Models Using OOD Examples](https://mlsanity.com/paper/2305.15269/) (2023)
- [Large Language Models for Generative Recommendation: A Survey and Visionary Discussions](https://mlsanity.com/paper/2309.01157/) (2023)
- [Interactive Path Reasoning on Graph for Conversational Recommendation](https://mlsanity.com/paper/2007.00194/) (2020)
- [Leveraging Large Language Models in Conversational Recommender Systems](https://mlsanity.com/paper/2305.07961/) (2023)
- [Unified Conversational Recommendation Policy Learning via Graph-based Reinforcement Learning](https://mlsanity.com/paper/2105.09710/) (2021)
- [Rethinking the Evaluation for Conversational Recommendation in the Era of Large Language Models](https://mlsanity.com/paper/2305.13112/) (2023)
- [BARCOR: Towards A Unified Framework for Conversational Recommendation Systems](https://mlsanity.com/paper/2203.14257/) (2022)

## Built on by (most cited first, 27 of 27)

- [Flipping the Dialogue: Training and Evaluating User Language Models](https://mlsanity.com/paper/2510.06552/) (2025)
- [A Survey of Large Language Model Empowered Agents for Recommendation and Search: Towards Next-Generation Information Retrieval](https://mlsanity.com/paper/2503.05659/) (2025)
- [LLM-based Human Simulations Have Not Yet Been Reliable](https://mlsanity.com/paper/2501.08579/) (2025)
- [Social Science Meets LLMs: How Reliable Are Large Language Models in Social Simulations?](https://mlsanity.com/paper/2410.23426/) (2024)
- [AgentCF++: Memory-enhanced LLM-based Agents for Popularity-aware Cross-domain Recommendations](https://mlsanity.com/paper/2502.13843/) (2025)
- [Large Language Models Do Not Simulate Human Psychology](https://mlsanity.com/paper/2508.06950/) (2025)
- [A LLM-based Controllable, Scalable, Human-Involved User Simulator Framework for Conversational Recommender Systems](https://mlsanity.com/paper/2405.08035/) (2024)
- [ConvApparel: A Benchmark Dataset and Validation Framework for User Simulators in Conversational Recommenders](https://mlsanity.com/paper/2602.16938/) (2026)
- [Knowledge is Power: Harnessing Large Language Models for Enhanced Cognitive Diagnosis](https://mlsanity.com/paper/2502.05556/) (2025)
- [Human vs. Agent in Task-Oriented Conversations](https://mlsanity.com/paper/2509.17619/) (2025)
- [One Battle After Another: Probing LLMs' Limits on Multi-Turn Instruction Following with a Benchmark Evolving Framework](https://mlsanity.com/paper/2511.03508/) (2025)
- [Survey of Computerized Adaptive Testing: A Machine Learning Perspective](https://mlsanity.com/paper/2404.00712/) (2024)
- [Reasoning over User Preferences: Knowledge Graph-Augmented LLMs for Explainable Conversational Recommendations](https://mlsanity.com/paper/2411.14459/) (2024)
- [Exploring Recommender System Evaluation: A Multi-Modal User Agent Framework for A/B Testing](https://mlsanity.com/paper/2601.04554/) (2026)
- [Autonomous Information Seeking: A Roadmap for Agentic Recommender Systems](https://mlsanity.com/paper/2607.04433/) (2026)
- [Analyzing and Internalizing Complex Policy Documents for LLM Agents](https://mlsanity.com/paper/2510.11588/) (2025)
- [ShiJianBench: From Dialogue to Decision for Long-Horizon Evaluation of Investment Advisors](https://mlsanity.com/paper/2608.01204/) (2026)
- [TestAgent: An Adaptive and Intelligent Expert for Human Assessment](https://mlsanity.com/paper/2506.03032/) (2025)
- [USB-Rec: An Effective Framework for Improving Conversational Recommendation Capability of Large Language Model](https://mlsanity.com/paper/2509.20381/) (2025)
- [Simulating Teams with LLM Agents: Interactive 2D Environments for Studying Human-AI Dynamics](https://mlsanity.com/paper/2510.08242/) (2025)
- [Interplay: Training Independent Simulators for Reference-Free Conversational Recommendation](https://mlsanity.com/paper/2603.18573/) (2026)
- [Personalized Turn-Level User Conversation Satisfaction Benchmark](https://mlsanity.com/paper/2605.29711/) (2026)
- [Beyond expert users: agents should help users construct preferences, not just elicit them](https://mlsanity.com/paper/2606.30863/) (2026)
- [Prompt Optimization for User Simulation in Conversational Recommender Systems: A Multi-Objective Framework](https://mlsanity.com/paper/2607.00010/) (2026)
- [Screen Before You Serve: Simulation for Production Customer Experience AI Agents at 140M Scale](https://mlsanity.com/paper/2609.30137/) (2026)
- [On Evaluating and Improving Conversational Agents in Production](https://mlsanity.com/paper/2609.32092/) (2026)
- [SimInter: Evidence-Grounded User Simulation with Interview-Based Profile Enrichment for Conversational Recommender Systems](https://mlsanity.com/paper/2610.18308/) (2026)

## Abstract (arXiv)

Conversational Recommender System (CRS) interacts with users through natural language to understand their preferences and provide personalized recommendations in real-time. CRS has demonstrated significant potential, prompting researchers to address the development of more realistic and reliable user simulators as a key focus. Recently, the capabilities of Large Language Models (LLMs) have attracted a lot of attention in various fields. Simultaneously, efforts are underway to construct user simulators based on LLMs. While these works showcase innovation, they also come with certain limitations that require attention. In this work, we aim to analyze the limitations of using LLMs in constructing user simulators for CRS, to guide future research. To achieve this goal, we conduct analytical validation on the notable work, iEvaLM. Through multiple experiments on two widely-used datasets in the field of conversational recommendation, we highlight several issues with the current evaluation methods for user simulators based on LLMs: (1) Data leakage, which occurs in conversational history and the user simulator's replies, results in inflated evaluation results. (2) The success of CRS recommendations depends more on the availability and quality of conversational history than on the responses from user simulators. (3) Controlling the output of the user simulator through a single prompt template proves challenging. To overcome these limitations, we propose SimpleUserSim, employing a straightforward strategy to guide the topic toward the target items. Our study validates the ability of CRS models to utilize the interaction information, significantly improving the recommendation results.