# LLMs struggle to simulate human belief updates in controlled environments

arXiv: https://arxiv.org/abs/2607.28347 · HTML: https://mlsanity.com/paper/2607.28347/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 23 of its 45 references that are in the map.
- First posted on arXiv 30 Jul 2026.

## Builds on (most cited first, 23 of 23)

- [The Llama 3 Herd of Models](https://mlsanity.com/paper/2407.21783/) (2024)
- [Generative Agents: Interactive Simulacra of Human Behavior](https://mlsanity.com/paper/2304.03442/) (2023)
- [A Survey on Large Language Model based Autonomous Agents](https://mlsanity.com/paper/2308.11432/) (2023)
- [τ-bench: A Benchmark for Tool-Agent-User Interaction in Real-World Domains](https://mlsanity.com/paper/2406.12045/) (2024)
- [Out of One, Many: Using Language Models to Simulate Human Samples](https://mlsanity.com/paper/2209.06899/) (2022)
- [τ2-Bench: Evaluating Conversational Agents in a Dual-Control Environment](https://mlsanity.com/paper/2506.07982/) (2025)
- [Large Language Models Empowered Agent-based Modeling and Simulation: A Survey and Perspectives](https://mlsanity.com/paper/2312.11970/) (2023)
- [PersonaLLM: Investigating the Ability of Large Language Models to Express Personality Traits](https://mlsanity.com/paper/2305.02547/) (2023)
- [AgentSociety: Large-Scale Simulation of LLM-Driven Generative Agents Advances Understanding of Human Behaviors and Society](https://mlsanity.com/paper/2502.08691/) (2025)
- [Simulating Opinion Dynamics with Networks of LLM-based Agents](https://mlsanity.com/paper/2311.09618/) (2023)
- [User Behavior Simulation with Large Language Model based Agents](https://mlsanity.com/paper/2306.02552/) (2023)
- [Evaluating and Inducing Personality in Pre-trained Language Models](https://mlsanity.com/paper/2206.07550/) (2022)
- [GenSim: A General Social Simulation Platform with Large Language Model based Agents](https://mlsanity.com/paper/2410.04360/) (2024)
- [Emergence of Social Norms in Generative Agent Societies: Principles and Architecture](https://mlsanity.com/paper/2403.08251/) (2024)
- [Decoding Echo Chambers: LLM-Powered Simulations Revealing Polarization in Social Networks](https://mlsanity.com/paper/2409.19338/) (2024)
- [SimUSER: Simulating User Behavior with Large Language Models for Recommender System Evaluation](https://mlsanity.com/paper/2504.12722/) (2025)
- [Emergence of polarization in networks of large language model agents](https://mlsanity.com/paper/2501.05171/) (2025)
- [GA-S3: Comprehensive Social Network Simulation with Group Agents](https://mlsanity.com/paper/2506.03532/) (2025)
- [Studying and improving reasoning in humans and machines](https://mlsanity.com/paper/2309.12485/) (2023)
- [Recognising, Anticipating, and Mitigating LLM Pollution of Online Behavioural Research](https://mlsanity.com/paper/2508.01390/) (2025)
- [Social Life Simulation for Non-Cognitive Skills Learning](https://mlsanity.com/paper/2405.00273/) (2024)
- [Value-Based Large Language Model Agent Simulation for Mutual Evaluation of Trust and Interpersonal Closeness](https://mlsanity.com/paper/2507.11979/) (2025)
- [Effects of personality steering on cooperative behavior in Large Language Model agents](https://mlsanity.com/paper/2601.05302/) (2026)

## Abstract (arXiv)

LLMs are increasingly deployed as proxies for human study participants in social science experiments, yet the fidelity of this practice has rarely been tested directly. We test whether six LLMs can simulate individual human belief updates, comparing LLM outputs 1-to-1 against ground truth data from 391 UK participants on Prolific, who updated their stances on three discussion topics after reading Reddit comments. Each participant was simulated by an LLM conditioned on a persona derived from their demographic and personality trait data. We find that some LLMs (Qwen3-32B and GPT-5-Mini) can match the human post-stance distribution, but only when given participants' actual initial stances. All six models fail to simulate initial stances themselves and to produce faithful belief updates from self-generated stances. Three systematic biases emerge across all models: overrepresentation of neutral positions, more frequent but smaller belief shifts than humans, and a failure to rank comments by convincingness. Demographic and personality trait personas had no consistent effect on fidelity. LLM simulations of human belief dynamics are only reliable when grounded in realistic starting conditions, that current multi-round social media simulations rarely provide.