mlsanityData to 7 Oct 2026

Towards Real-world Human Behavior Simulation: Benchmarking Large Language Models on Long-horizon, Cross-scenario, Heterogeneous Behavior Traces

arXiv 2604.08362 ↗First posted 9 Apr 2026cs.CL cs.AI cs.LGcited by 6

Not on the map. Why?

Abstract (arXiv)

The emergence of Large Language Models (LLMs) has illuminated the potential for a general-purpose user simulator. However, existing benchmarks remain constrained to isolated scenarios, narrow action spaces, or synthetic data, failing to capture the holistic nature of authentic human behavior. To bridge this gap, we introduce OmniBehavior, the first user simulation benchmark constructed entirely from real-world data, integrating long-horizon, cross-scenario, and heterogeneous behavioral patterns into a unified framework. Based on this benchmark, we first provide empirical evidence that previous datasets with isolated scenarios suffer from tunnel vision, whereas real-world decision-making relies on long-term, cross-scenario causal chains. Extensive evaluations of state-of-the-art LLMs reveal that current models struggle to accurately simulate these complex behaviors, with performance plateauing even as context windows expand. Crucially, a systematic comparison between simulated and authentic behaviors uncovers a fundamental structural bias: LLMs tend to converge toward a positive average person, exhibiting hyper-activity, persona homogenization, and a utopian bias. This results in the loss of individual differences and long-tail behaviors, highlighting critical directions for future high-fidelity simulation research.

Key facts. Cited by 6 papers in the map (data through 7 Oct 2026). Builds on 35 of its 56 references that are in the map. First posted on arXiv 9 Apr 2026.

Builds on

35 of 56 references.

References · 35

  1. 2023GPT-4 Technical Reportcited by 18,132
  2. 2025Qwen3 Technical Reportcited by 8,195
  3. 2023Efficient Memory Management for Large Language Model Serving with PagedAttentioncited by 5,767
  4. 2024GPT-4o System Cardcited by 4,467
  5. 2023Generative Agents: Interactive Simulacra of Human Behaviorcited by 3,500
  6. 2023Lost in the Middle: How Language Models Use Long Contextscited by 3,383
  7. 2023Towards Understanding Sycophancy in Language Modelscited by 988
  8. 2024Evaluating Very Long-Term Conversational Memory of LLM Agentscited by 800
  9. 2023ChatEval: Towards Better LLM-based Evaluators through Multi-Agent Debatecited by 724
  10. 2017Deep Interest Network for Click-Through Rate Predictioncited by 541
  11. 2023Large Language Models Empowered Agent-based Modeling and Simulation: A Survey and Perspectivescited by 253
  12. 2020Search-based User Interest Modeling with Lifelong Sequential Behavior Data for Click-Through Rate Predictioncited by 176
23 more
  1. 2023RoleLLM: Benchmarking, Eliciting, and Enhancing Role-Playing Abilities of Large Language Modelscited by 173
  2. 2025Qwen2.5-1M Technical Reportcited by 158
  3. 2023AgentCF: Collaborative Learning with Autonomous Language Agents for Recommender Systemscited by 116
  4. 2013A Computational Approach to Politeness with Application to Social Factorscited by 104
  5. 2022KuaiRec: A Fully-observed Dataset and Insights for Evaluating Recommender Systemscited by 94
  6. 2019RecSim: A Configurable Simulation Platform for Recommender Systemscited by 91
  7. 2018Virtual-Taobao: Virtualizing Real-world Online Retail Environment for Reinforcement Learningcited by 80
  8. 2024Unveiling the Truth and Facilitating Change: Towards Agent-based Large-scale Social Movement Simulationcited by 69
  9. 2025SimBench: Benchmarking the Ability of Large Language Models to Simulate Human Behaviorscited by 48
  10. 2020Evaluating Conversational Recommender Systems via User Simulationcited by 46
  11. 2024Evaluating Large Language Models as Generative User Simulators for Conversational Recommendationcited by 45
  12. 2024CharacterBox: Evaluating the Role-Playing Capabilities of LLMs in Text-Based Virtual Worldscited by 38
  13. 2025SimUSER: Simulating User Behavior with Large Language Models for Recommender System Evaluationcited by 32
  14. 2022Tenrec: A Large-scale Multipurpose Benchmark Dataset for Recommender Systemscited by 31
  15. 2024BASES: Large-scale Web Search User Simulation with Large Language Model based Agentscited by 24
  16. 2024Human Simulacra: Benchmarking the Personification of Large Language Modelscited by 17
  17. 2024Computational Politeness in Natural Language Processing: A Surveycited by 8
  18. 2025From the Fluency Fallacy to the Micro-to-Macro Validity Gap: Opportunities and Pitfalls of LLMs in Social Simulationcited by 8
  19. 2021Simulations in Recommender Systems: An industry perspectivecited by 6
  20. 2021The JDDC 2.0 Corpus: A Large-Scale Multimodal Multi-Turn Chinese Dialogue Dataset for E-commerce Customer Servicecited by 5
  21. 2025KuaiLive: A Real-time Interactive Dataset for Live Streaming Recommendationcited by 5
  22. 2025AI-Salesman: Towards Reliable Large Language Model Driven Telemarketingcited by 2
  23. 2025Sim4IA-Bench: A User Simulation Benchmark Suite for Next Query and Utterance Predictioncited by 1

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2026: 6.

6 papers.