mlsanityData to 14 Oct 2026

SimUSER: Simulating User Behavior with Large Language Models for Recommender System Evaluation

arXiv 2504.12722 ↗First posted 17 Apr 2025cs.IR cs.AIcited by 33

Not on the map. Why?

Abstract (arXiv)

Recommender systems play a central role in numerous real-life applications, yet evaluating their performance remains a significant challenge due to the gap between offline metrics and online behaviors. Given the scarcity and limits (e.g., privacy issues) of real user data, we introduce SimUSER, an agent framework that serves as believable and cost-effective human proxies. SimUSER first identifies self-consistent personas from historical data, enriching user profiles with unique backgrounds and personalities. Then, central to this evaluation are users equipped with persona, memory, perception, and brain modules, engaging in interactions with the recommender system. SimUSER exhibits closer alignment with genuine humans than prior work, both at micro and macro levels. Additionally, we conduct insightful experiments to explore the effects of thumbnails on click rates, the exposure effect, and the impact of reviews on user engagement. Finally, we refine recommender system parameters based on offline A/B test results, resulting in improved user engagement in the real world.

Key facts. Cited by 33 papers in the map (data through 14 Oct 2026). Builds on 18 of its 31 references that are in the map. First posted on arXiv 17 Apr 2025.

Builds on

18 of 31 references.

References · 18

  1. 2020LightGCN: Simplifying and Powering Graph Convolution Network for Recommendationcited by 1,139
  2. 2023Can Large Language Models Be an Alternative to Human Evaluations?cited by 734
  3. 2018Variational Autoencoders for Collaborative Filteringcited by 310
  4. 2023Large Language Models are Zero-Shot Rankers for Recommender Systemscited by 309
  5. 2023Recommender AI Agent: Integrating Large Language Models for Interactive Recommendationscited by 131
  6. 2023Large Language Models as Zero-Shot Conversational Recommenderscited by 127
  7. 2017Attentional Factorization Machines: Learning the Weight of Feature Interactions via Attention Networkscited by 126
  8. 2023Do LLMs Understand User Preferences? Evaluating LLMs On User Rating Predictioncited by 116
  9. 2023AgentCF: Collaborative Learning with Autonomous Language Agents for Recommender Systemscited by 116
  10. 2023GPT4Rec: A Generative Framework for Personalized Recommendation and User Interests Interpretationcited by 103
  11. 2020Estimation-Action-Reflection: Towards Deep Interaction Between Conversational and Recommender Systemscited by 98
  12. 2024Evaluating Large Language Models as Generative User Simulators for Conversational Recommendationcited by 48
6 more
  1. 2019Measuring the Business Value of Recommender Systemscited by 32
  2. 2022Tenrec: A Large-scale Multipurpose Benchmark Dataset for Recommender Systemscited by 31
  3. 2023KuaiSim: A Comprehensive Simulator for Recommender Systemscited by 30
  4. 2024LLM-Enhanced User-Item Interactions: Leveraging Edge Information for Optimized Recommendationscited by 24
  5. 2024Prospect Personalized Recommendation on Large Language Model-based Agent Platformcited by 19
  6. 2024Rec-GPT4V: Multimodal Recommendation with Large Vision-Language Modelscited by 18

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2025: 7; 2026: 26.

33 papers.

Sort:

By area:

  1. 2026Mind the Sim2Real Gap in User Simulation for Agentic Taskscited by 54
  2. 2025CitySim: Modeling Urban Behaviors and City Dynamics with Large-Scale LLM-Driven Agent Simulationcited by 28
  3. 2025RecoWorld: Building Simulated Environments for Agentic Recommender Systemscited by 19
  4. 2026RealUserSim: Bridging the Reality Gap in Agent Benchmarking via Grounded User Simulationcited by 12
  5. 2026Towards Real-world Human Behavior Simulation: Benchmarking Large Language Models on Long-horizon, Cross-scenario, Heterogeneous Behavior Tracescited by 9
  6. 2026MirrorBench: A Benchmark to Evaluate Conversational User-Proxy Agents for Human-Likenesscited by 7
  7. 2025SAGE: A Top-Down Bottom-Up Knowledge-Grounded User Simulator for Multi-turn AGent Evaluationcited by 6
  8. 2026AlignUSER: Human-Aligned LLM Agents via World Models for Recommender System Evaluationcited by 6
  9. 2025Beyond Static Testbeds: An Interaction-Centric Agent Simulation Platform for Dynamic Recommender Systemscited by 5
  10. 2026Agentic Recommender System with Hierarchical Belief-State Memorycited by 5
  11. 2026HyTRec: A Hybrid Temporal-Aware Attention Architecture for Long Behavior Sequential Recommendationcited by 4
  12. 2026Reinforcing Human Behavior Simulation via Verbal Feedbackcited by 4
21 more
  1. 2026DynamicMem: A Long-Horizon Memory Benchmark in Real-World Settingscited by 4
  2. 2025GGBond: Growing Graph-Based AI-Agent Society for Socially-Aware Recommender Simulationcited by 3
  3. 2025A Comprehensive Review on Harnessing Large Language Models to Overcome Recommender System Challengescited by 3
  4. 2026Beyond Offline A/B Testing: Context-Aware Agent Simulation for Recommender System Evaluationcited by 3
  5. 2025ParaStudent: Closing the Sim2Real Gap in User Simulators for AI Tutor Evaluationcited by 2
  6. 2026Breaking User-Centric Agency: A Tri-Party Framework for Agent-Based Recommendationcited by 2
  7. 2026How Well Do Large Language Models Capture Human Personality?cited by 2
  8. 2026CORAL: An LLM-Native Harness for Production Recommender Systemscited by 2
  9. 2026ShiJianBench: From Dialogue to Decision for Long-Horizon Evaluation of Investment Advisorscited by 1
  10. 2026Learning from Online User Feedback for Shopping Agentscited by 1
  11. 2026PersonaAct: Simulating Short-Video Users with Personalized Agents for Counterfactual Filter Bubble Auditingcited by 0
  12. 2026NextAds: Towards Next-generation Personalized Video Advertisingcited by 0
  13. 2026Through Their Eyes: Fixation-aligned Tuning for Personalized User Emulationcited by 0
  14. 2026PERCEIVE: A Benchmark for Personalized Emotion and Communication Behavior Understanding on Social Mediacited by 0
  15. 2026SR-Agent: An Experience-Driven Agentic Framework for Post-Ranking Strategy Refinement in E-Commerce Recommendationcited by 0
  16. 2026LLMs struggle to simulate human belief updates in controlled environmentscited by 0
  17. 2026CityReal: Human-Aligned Urban Behavior and City Dynamics Simulation with Large-Scale LLM Agentscited by 0
  18. 2026Ask or Answer: A Decision Framework for Multi-Turn Health Misinformation Interventioncited by 0
  19. 2026LiveSim: Simulating Environment-Shaped Users in Multi-Agent Live-Stream Ecosystemscited by 0
  20. 2026A Behavioral Trait Leaks into Preferences: Diagnosing Trait Interference in LLM User Simulatorscited by 0
  21. 2026PsyBer-Agent: A Psychology-Driven User Agent for Recommender Systemscited by 0