OdysSim: Building Foundation Models for Human Behavior Simulation
Not on the map. Why?
Abstract (arXiv)
Large language models are increasingly deployed as human simulators for interactive evaluation and social simulation. Yet helpfulness-driven post-training pulls them toward a homogeneous, overly agreeable assistant register, creating a behavioral Sim2Real gap. We present OdysSim, the largest open systematic investigation of behavioral foundation models, i.e., models trained to simulate human behavior at scale. We propose SOUL, a taxonomy of five capability axes (CONV, SS, COG, ROLE, EVAL) that unifies 62 datasets and 23 benchmark tasks under one framework. Specifically, we curate the OdysSim corpus (21.4M interactions, 10B tokens, retrofitted with back-generated social contexts), construct the SOUL-Index benchmark, and develop an end-to-end training recipe combining midtraining, task-specific RL, and expert distillation. The resulting open 8B OSim model ranks first or tied-first on 8 of 23 tasks, outperforming any individual frontier model by this count, with the strongest gains on conversational and social tasks. Its outputs are also more human-like in length, formatting, and word choice, and it transfers zero-shot to out-of-distribution user simulation on τ-bench, nearly matching real users on reaction alignment (93.2 vs. 93.5). We further show that LLM-as-judge RL induces reward-hacking patterns, and that our detectors can mitigate them during post-training. Together, our findings suggest that behavioral foundation models require rethinking the LLM training paradigm. We release all artifacts to support future research.
Key facts. Cited by 8 papers in the map (data through 7 Oct 2026). Builds on 46 of its 59 references that are in the map. First posted on arXiv 12 Jun 2026.
Builds on
46 of 59 references.
References · 46
- 2022Training language models to follow instructions with human feedbackcited by 15,712
- 2023Judging LLM-as-a-Judge with MT-Bench and Chatbot Arenacited by 8,393
- 2023Generative Agents: Interactive Simulacra of Human Behaviorcited by 3,500
- 2020Don't Stop Pretraining: Adapt Language Models to Domains and Taskscited by 1,548
- 2024τ-bench: A Benchmark for Tool-Agent-User Interaction in Real-World Domainscited by 1,280
- 2022Out of One, Many: Using Language Models to Simulate Human Samplescited by 737
- 2017DailyDialog: A Manually Labelled Multi-turn Dialogue Datasetcited by 649
- 2023LMSYS-Chat-1M: A Large-Scale Real-World LLM Conversation Datasetcited by 449
- 2023SOTOPIA: Interactive Evaluation for Social Intelligence in Language Agentscited by 350
- 2025Persona Vectors: Monitoring and Controlling Character Traits in Language Modelscited by 303
- 2024Replacing Judges with Juries: Evaluating LLM Generations with a Panel of Diverse Modelscited by 301
- 2021Towards Emotional Support Dialog Systemscited by 294
34 more
- 2026Reinforcement Learning via Self-Distillationcited by 285
- 2019Persuasion for Good: Towards a Personalized Persuasive Dialogue System for Social Goodcited by 207
- 2023FANToM: A Benchmark for Stress-testing Machine Theory of Mind in Interactionscited by 162
- 2025Artificial Hivemind: The Open-Ended Homogeneity of Language Models (and Beyond)cited by 137
- 2024The PRISM Alignment Dataset: What Participatory, Representative and Individualised Human Feedback Reveals About the Subjective and Multicultural Alignment of Large Language Modelscited by 136
- 2020Moral Stories: Situated Reasoning about Norms, Intents, Actions, and their Consequencescited by 133
- 2023Minding Language Models' (Lack of) Theory of Mind: A Plug-and-Play Multi-Character Belief Trackercited by 95
- 2024SOTOPIA-π: Interactive Learning of Socially Intelligent Language Agentscited by 89
- 2020ConvoKit: A Toolkit for the Analysis of Conversationscited by 71
- 2023HI-TOM: A Benchmark for Evaluating Higher-Order Theory of Mind Reasoning in Large Language Modelscited by 61
- 2021CaSiNo: A Corpus of Campsite Negotiation Dialogues for Automatic Negotiation Systemscited by 60
- 2025Zero-Shot Whole-Body Humanoid Control via Behavioral Foundation Modelscited by 51
- 2025Flipping the Dialogue: Training and Evaluating User Language Modelscited by 50
- 2024BIG5-CHAT: Shaping LLM Personalities Through Training on Human-Grounded Datacited by 49
- 2026Expanding the Capabilities of Reinforcement Learning via Text Feedbackcited by 49
- 2025SimulatorArena: Are User Simulators Reliable Proxies for Multi-Turn Evaluation of AI Assistants?cited by 45
- 2026Mind the Sim2Real Gap in User Simulation for Agentic Taskscited by 44
- 2024HAICOSYSTEM: An Ecosystem for Sandboxing Safety Risks in Human-AI Interactionscited by 42
- 2025Finetuning LLMs for Human Behavior Prediction in Social Science Experimentscited by 35
- 2024Character is Destiny: Can Role-Playing Language Agents Make Persona-Driven Decisions?cited by 34
- 2026HumanLM: Simulating Users with State Alignment Beats Response Imitationcited by 31
- 2025PatientSim: A Persona-Driven Simulator for Realistic Doctor-Patient Interactionscited by 30
- 2025From 1,000,000 Users to Every User: Scaling Up Personalized Preference for User-level Alignmentcited by 28
- 2025CoSER: A Comprehensive Literary Dataset and Framework for Training and Evaluating LLM Role-Playing and Persona Simulationcited by 26
- 2026Experiential Reinforcement Learningcited by 21
- 2025Learning to Make MISTAKEs: Modeling Incorrect Student Thinking And Key Errorscited by 19
- 2023MindGames: Targeting Theory of Mind in Large Language Models with Dynamic Epistemic Modal Logiccited by 18
- 2025Mid-Training of Large Language Models: A Surveycited by 18
- 2025TwinVoice: A Multi-dimensional Benchmark Towards Digital Twins via LLM Persona Simulationcited by 13
- 2025HumT DumT: Measuring and controlling human-like language in LLMscited by 11
- 2026MirrorBench: A Benchmark to Evaluate Conversational User-Proxy Agents for Human-Likenesscited by 7
- 2026One Model, All Roles: Multi-Turn, Multi-Agent Self-Play Reinforcement Learning for Conversational Social Intelligencecited by 7
- 2026Social-R1: Towards Human-like Social Reasoning in LLMscited by 4
- 2026Reinforcing Human Behavior Simulation via Verbal Feedbackcited by 4
Built on by
Citing papers per year: 2026: 8.
8 papers.
- 2026Learning User Simulators with Turing Rewardscited by 3
- 2026BehaviorBench: Benchmarking Foundation Models for Behavioral Science Taskscited by 0
- 2026Readable, Faithful, Used: Three Dissociable Properties of Demographic Identity in a Language Modelcited by 0
- 2026Mind2Dialogue: Training Human-Aware Language Models by Simulating User Mental Statescited by 0
- 2026Verifiable Social Reasoning for LLM Assistantscited by 0
- 2026From Learner Behavior to Reusable Skills for Effective and Efficient Learner Simulationcited by 0
- 2026AnthroDial: Benchmarking LLM Anthropomorphism in Autonomous Social Interactioncited by 0
- 2026MIMESIS: Learning User Simulators as Training Environments for Interactive Agentscited by 0
- 2026MIMESIS: Learning User Simulators as Training Environments for Interactive Agents7 Oct 2026
- 2026AnthroDial: Benchmarking LLM Anthropomorphism in Autonomous Social Interaction29 Sep 2026
- 2026From Learner Behavior to Reusable Skills for Effective and Efficient Learner Simulation29 Sep 2026
- 2026Verifiable Social Reasoning for LLM Assistants15 Sep 2026
- 2026Mind2Dialogue: Training Human-Aware Language Models by Simulating User Mental States14 Sep 2026
- 2026Readable, Faithful, Used: Three Dissociable Properties of Demographic Identity in a Language Model19 Aug 2026
- 2026BehaviorBench: Benchmarking Foundation Models for Behavioral Science Tasks23 Jun 2026
- 2026Learning User Simulators with Turing Rewards17 Jun 2026