# Large Language Models Do Not Simulate Human Psychology

arXiv: https://arxiv.org/abs/2508.06950 · HTML: https://mlsanity.com/paper/2508.06950/

## Key facts

- Cited by 19 papers in the map (data through 14 Oct 2026).
- Builds on 17 of its 45 references that are in the map.
- First posted on arXiv 9 Aug 2025.

## Builds on (most cited first, 17 of 17)

- [Training language models to follow instructions with human feedback](https://mlsanity.com/paper/2203.02155/) (2022)
- [Generative Agents: Interactive Simulacra of Human Behavior](https://mlsanity.com/paper/2304.03442/) (2023)
- [Sparks of Artificial General Intelligence: Early experiments with GPT-4](https://mlsanity.com/paper/2303.12712/) (2023)
- [Bias and Fairness in Large Language Models: A Survey](https://mlsanity.com/paper/2309.00770/) (2023)
- [Instruction Tuning for Large Language Models: A Survey](https://mlsanity.com/paper/2308.10792/) (2023)
- [LLM Agents Grounded in Self-Reports Enable General-Purpose Simulation of Individuals](https://mlsanity.com/paper/2411.10109/) (2024)
- [Evaluating Large Language Models in Theory of Mind Tasks](https://mlsanity.com/paper/2302.02083/) (2023)
- [Language models are not naysayers: An analysis of language models on negation benchmarks](https://mlsanity.com/paper/2306.08189/) (2023)
- [Exploring the psychology of LLMs' Moral and Legal Reasoning](https://mlsanity.com/paper/2308.01264/) (2023)
- [Evaluating Cultural Adaptability of a Large Language Model via Simulation of Synthetic Personas](https://mlsanity.com/paper/2408.06929/) (2024)
- [LLM-based Human Simulations Have Not Yet Been Reliable](https://mlsanity.com/paper/2501.08579/) (2025)
- [How Reliable is Your Simulator? Analysis on the Limitations of Current LLM-based User Simulators for Conversational Recommendation](https://mlsanity.com/paper/2403.16416/) (2024)
- [Re-evaluating Theory of Mind evaluation in large language models](https://mlsanity.com/paper/2502.21098/) (2025)
- [Attributions toward Artificial Agents in a modified Moral Turing Test](https://mlsanity.com/paper/2406.11854/) (2024)
- [The Moral Turing Test: Evaluating Human-LLM Alignment in Moral Decision-Making](https://mlsanity.com/paper/2410.07304/) (2024)
- [Can Machines Think Like Humans? A Behavioral Evaluation of LLM Agents in Dictator Games](https://mlsanity.com/paper/2410.21359/) (2024)
- [Fine-Grained Detection of Solidarity for Women and Migrants in 155 Years of German Parliamentary Debates](https://mlsanity.com/paper/2210.04359/) (2022)

## Built on by (most cited first, 19 of 19)

- [Moloch's Bargain: Emergent Misalignment When LLMs Compete for Audiences](https://mlsanity.com/paper/2510.06105/) (2025)
- [Take Out Your Calculators: Estimating the Real Difficulty of Question Items with LLM Student Simulations](https://mlsanity.com/paper/2601.09953/) (2026)
- [Addressing Longstanding Challenges in Cognitive Science with Language Models](https://mlsanity.com/paper/2511.00206/) (2025)
- [The Judgment-Consequence Gap: LLM Moral Reasoning in Healthcare Decisions](https://mlsanity.com/paper/2608.05583/) (2026)
- [Can Finetuing LLMs on Small Human Samples Increase Heterogeneity, Alignment, and Belief-Action Coherence?](https://mlsanity.com/paper/2511.21218/) (2025)
- [HumanLLM: Benchmarking and Improving LLM Anthropomorphism via Human Cognitive Patterns](https://mlsanity.com/paper/2601.10198/) (2026)
- [Dynamic Personality Adaptation in Large Language Models via State Machines](https://mlsanity.com/paper/2602.22157/) (2026)
- [GTA: Generative Traffic Agents for Simulating Realistic Mobility Behavior](https://mlsanity.com/paper/2601.16778/) (2026)
- [LLM surprisal is necessary but not sufficient to capture English garden-path effects: Evidence from joint latent modeling of reading paradigms](https://mlsanity.com/paper/2602.04489/) (2026)
- [Interview-Informed Generative Agents for Product Discovery: A Validation Study](https://mlsanity.com/paper/2603.29890/) (2026)
- [LifeSide: Benchmarking Agents as Lifelong Digital Companions](https://mlsanity.com/paper/2606.04660/) (2026)
- [Measure what Matters: Psychometric Evaluation of AI with Situational Judgment Tests](https://mlsanity.com/paper/2510.22170/) (2025)
- [The Third Ambition: Artificial Intelligence and the Science of Human Behavior](https://mlsanity.com/paper/2603.07329/) (2026)
- [The Storyteller in the Model: Narrative Pattern Inheritance, Escalation Dynamics, and Alignment Governance in LLMs](https://mlsanity.com/paper/2607.20449/) (2026)
- [Position: We Need Practical AI Alignment Methods to Mirror Human Reasoning](https://mlsanity.com/paper/2608.12372/) (2026)
- [Process-Constituted Intelligence: A Shared Criterion for Humans and Machines](https://mlsanity.com/paper/2608.16213/) (2026)
- [Aligned but Flattened: Analyzing the Trade-off between Cultural Alignment and Diversity in LLMs](https://mlsanity.com/paper/2609.00565/) (2026)
- [Aligning with Lived Experience: Heterogeneous Benefits of Fine Tuning in Mental Health Support Generation](https://mlsanity.com/paper/2609.21075/) (2026)
- [Moral Combat: Exploiting Moral Foundations Theory to Jailbreak LLMs](https://mlsanity.com/paper/2610.19587/) (2026)

## Abstract (arXiv)

Large Language Models (LLMs),such as ChatGPT, are increasingly used in research, ranging from simple writing assistance to complex data annotation tasks. Recently, some research has suggested that LLMs may even be able to simulate human psychology and can, hence, replace human participants in psychological studies. We caution against this approach. We provide conceptual arguments against the hypothesis that LLMs simulate human psychology. We then present empiric evidence illustrating our arguments by demonstrating that slight changes to wording that correspond to large changes in meaning lead to notable discrepancies between LLMs' and human responses, even for the recent CENTAUR model that was specifically fine-tuned on psychological responses. Additionally, different LLMs show very different responses to novel items, further illustrating their lack of reliability. We conclude that LLMs do not simulate human psychology and recommend that psychological researchers should treat LLMs as useful but fundamentally unreliable tools that need to be validated against human responses for every new application.