Towards Real-world Human Behavior Simulation: Benchmarking Large Language Models on Long-horizon, Cross-scenario, Heterogeneous Behavior Traces
Not on the map. Why?
Abstract (arXiv)
The emergence of Large Language Models (LLMs) has illuminated the potential for a general-purpose user simulator. However, existing benchmarks remain constrained to isolated scenarios, narrow action spaces, or synthetic data, failing to capture the holistic nature of authentic human behavior. To bridge this gap, we introduce OmniBehavior, the first user simulation benchmark constructed entirely from real-world data, integrating long-horizon, cross-scenario, and heterogeneous behavioral patterns into a unified framework. Based on this benchmark, we first provide empirical evidence that previous datasets with isolated scenarios suffer from tunnel vision, whereas real-world decision-making relies on long-term, cross-scenario causal chains. Extensive evaluations of state-of-the-art LLMs reveal that current models struggle to accurately simulate these complex behaviors, with performance plateauing even as context windows expand. Crucially, a systematic comparison between simulated and authentic behaviors uncovers a fundamental structural bias: LLMs tend to converge toward a positive average person, exhibiting hyper-activity, persona homogenization, and a utopian bias. This results in the loss of individual differences and long-tail behaviors, highlighting critical directions for future high-fidelity simulation research.
Key facts. Cited by 6 papers in the map (data through 7 Oct 2026). Builds on 35 of its 56 references that are in the map. First posted on arXiv 9 Apr 2026.
Builds on
35 of 56 references.
References · 35
- 2023GPT-4 Technical Reportcited by 18,132
- 2025Qwen3 Technical Reportcited by 8,195
- 2023Efficient Memory Management for Large Language Model Serving with PagedAttentioncited by 5,767
- 2024GPT-4o System Cardcited by 4,467
- 2023Generative Agents: Interactive Simulacra of Human Behaviorcited by 3,500
- 2023Lost in the Middle: How Language Models Use Long Contextscited by 3,383
- 2023Towards Understanding Sycophancy in Language Modelscited by 988
- 2024Evaluating Very Long-Term Conversational Memory of LLM Agentscited by 800
- 2023ChatEval: Towards Better LLM-based Evaluators through Multi-Agent Debatecited by 724
- 2017Deep Interest Network for Click-Through Rate Predictioncited by 541
- 2023Large Language Models Empowered Agent-based Modeling and Simulation: A Survey and Perspectivescited by 253
- 2020Search-based User Interest Modeling with Lifelong Sequential Behavior Data for Click-Through Rate Predictioncited by 176
23 more
- 2023RoleLLM: Benchmarking, Eliciting, and Enhancing Role-Playing Abilities of Large Language Modelscited by 173
- 2025Qwen2.5-1M Technical Reportcited by 158
- 2023AgentCF: Collaborative Learning with Autonomous Language Agents for Recommender Systemscited by 116
- 2013A Computational Approach to Politeness with Application to Social Factorscited by 104
- 2022KuaiRec: A Fully-observed Dataset and Insights for Evaluating Recommender Systemscited by 94
- 2019RecSim: A Configurable Simulation Platform for Recommender Systemscited by 91
- 2018Virtual-Taobao: Virtualizing Real-world Online Retail Environment for Reinforcement Learningcited by 80
- 2024Unveiling the Truth and Facilitating Change: Towards Agent-based Large-scale Social Movement Simulationcited by 69
- 2025SimBench: Benchmarking the Ability of Large Language Models to Simulate Human Behaviorscited by 48
- 2020Evaluating Conversational Recommender Systems via User Simulationcited by 46
- 2024Evaluating Large Language Models as Generative User Simulators for Conversational Recommendationcited by 45
- 2024CharacterBox: Evaluating the Role-Playing Capabilities of LLMs in Text-Based Virtual Worldscited by 38
- 2025SimUSER: Simulating User Behavior with Large Language Models for Recommender System Evaluationcited by 32
- 2022Tenrec: A Large-scale Multipurpose Benchmark Dataset for Recommender Systemscited by 31
- 2024BASES: Large-scale Web Search User Simulation with Large Language Model based Agentscited by 24
- 2024Human Simulacra: Benchmarking the Personification of Large Language Modelscited by 17
- 2024Computational Politeness in Natural Language Processing: A Surveycited by 8
- 2025From the Fluency Fallacy to the Micro-to-Macro Validity Gap: Opportunities and Pitfalls of LLMs in Social Simulationcited by 8
- 2021Simulations in Recommender Systems: An industry perspectivecited by 6
- 2021The JDDC 2.0 Corpus: A Large-Scale Multimodal Multi-Turn Chinese Dialogue Dataset for E-commerce Customer Servicecited by 5
- 2025KuaiLive: A Real-time Interactive Dataset for Live Streaming Recommendationcited by 5
- 2025AI-Salesman: Towards Reliable Large Language Model Driven Telemarketingcited by 2
- 2025Sim4IA-Bench: A User Simulation Benchmark Suite for Next Query and Utterance Predictioncited by 1
Built on by
Citing papers per year: 2026: 6.
6 papers.
- 2026Vision-OPD: Learning to See Fine Details for Multimodal LLMs via On-Policy Self-Distillationcited by 71
- 2026Measuring and Mitigating the Distributional Gap Between Real and Simulated User Behaviorscited by 4
- 2026The Deliberative Deficit: An Empirical Critique of LLMs in Democratic Discoursecited by 2
- 2026PraMem: Practice-derived Experiential Memory for Long-horizon Behavior Predictioncited by 0
- 2026Measuring the Behavioral Fidelity of Long-Horizon Human Activity Simulationscited by 0
- 2026Are LLMs Good Financial User Simulators? Multi-view Investor Logic Alignment (MILA)cited by 0
- 2026Are LLMs Good Financial User Simulators? Multi-view Investor Logic Alignment (MILA)14 Sep 2026
- 2026Measuring the Behavioral Fidelity of Long-Horizon Human Activity Simulations1 Sep 2026
- 2026The Deliberative Deficit: An Empirical Critique of LLMs in Democratic Discourse10 Aug 2026
- 2026PraMem: Practice-derived Experiential Memory for Long-horizon Behavior Prediction3 Jul 2026
- 2026Vision-OPD: Learning to See Fine Details for Multimodal LLMs via On-Policy Self-Distillation18 May 2026
- 2026Measuring and Mitigating the Distributional Gap Between Real and Simulated User Behaviors8 May 2026