# Six Fallacies in Substituting Large Language Models for Human Participants

arXiv: https://arxiv.org/abs/2402.04470 · HTML: https://mlsanity.com/paper/2402.04470/

## Key facts

- Cited by 24 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 6 Feb 2024.

## Built on by (most cited first, 24 of 24)

- [The threat of analytic flexibility in using large language models to simulate human data](https://mlsanity.com/paper/2509.13397/) (2025)
- [From Prompts to Constructs: A Dual-Validity Framework for Large Language Model Research in Psychology](https://mlsanity.com/paper/2506.16697/) (2025)
- [A validity-guided workflow for robust large language model research in psychology](https://mlsanity.com/paper/2507.04491/) (2025)
- [Large Language Models as Psychological Simulators: A Methodological Guide](https://mlsanity.com/paper/2506.16702/) (2025)
- [When Synthetic Users Fail: A Cross-Domain Benchmark of LLM-Simulated Human Survey Responses](https://mlsanity.com/paper/2607.26348/) (2026)
- [Adding LLMs to the psycholinguistic norming toolbox: A practical guide to getting the most out of human ratings](https://mlsanity.com/paper/2509.14405/) (2025)
- [Assessing Risks of Large Language Models in Mental Health Support: A Framework for Automated Clinical AI Red Teaming](https://mlsanity.com/paper/2602.19948/) (2026)
- [Towards Operational Validation of LLM-Agent Social Simulations: A Replicated Study of a Reddit-like Technology Forum](https://mlsanity.com/paper/2508.21740/) (2025)
- [Can Finetuing LLMs on Small Human Samples Increase Heterogeneity, Alignment, and Belief-Action Coherence?](https://mlsanity.com/paper/2511.21218/) (2025)
- [Debiasing International Attitudes: LLM Agents for Simulating US-China Perception Changes](https://mlsanity.com/paper/2508.08837/) (2025)
- [LLM surprisal is necessary but not sufficient to capture English garden-path effects: Evidence from joint latent modeling of reading paradigms](https://mlsanity.com/paper/2602.04489/) (2026)
- [The yes-no bias of large language models reflects answer order and wording, not shifts in moral judgment](https://mlsanity.com/paper/2607.05552/) (2026)
- [Item-Mean Surrogates: Why Richer Persona Data Fail to Improve LLMs as Human Surrogates](https://mlsanity.com/paper/2608.29455/) (2026)
- [Consequential Behaviour and Representational Fairness in the Validation of Synthetic Research](https://mlsanity.com/paper/2609.27690/) (2026)
- [Beyond Instrumental and Substitutive Paradigms: Introducing Machine Culture as an Emergent Phenomenon in Large Language Models](https://mlsanity.com/paper/2601.17096/) (2026)
- [Chinese sensorimotor and embodiment norms for 3,000 lexicalized concepts](https://mlsanity.com/paper/2605.22616/) (2026)
- [APS: Bias-Controlled Adaptive Prototype Simulation for Population-Scale LLM Agents](https://mlsanity.com/paper/2605.27419/) (2026)
- [How Do Large Language Models Judge Social Attraction? Evidence from Theory-Grounded Persona Ratings Across Multiple LLMs and Humans](https://mlsanity.com/paper/2608.09717/) (2026)
- [Process-Constituted Intelligence: A Shared Criterion for Humans and Machines](https://mlsanity.com/paper/2608.16213/) (2026)
- [Six misconceptions about large language models: A minimal model and diagnostic taxonomy](https://mlsanity.com/paper/2608.20421/) (2026)
- [Human-like moral judgments conceal divergent motive attributions in large language models](https://mlsanity.com/paper/2609.07353/) (2026)
- [The Internal Anatomy of Strategic Choice in Large Language Models](https://mlsanity.com/paper/2609.07478/) (2026)
- [From Process Loss to Assembly Bonus: Human-Grounded Diagnosis of Multi-Agent LLM Collaboration](https://mlsanity.com/paper/2609.13261/) (2026)
- [Thinking Less to Simulate Better: Intuitive Prompting Improves LLM Agents Simulating Individual Social Media Reactions, Including Unfamiliar Content](https://mlsanity.com/paper/2609.30563/) (2026)

## Abstract (arXiv)

Can AI systems like large language models (LLMs) replace human participants in behavioral and psychological research? Here I critically evaluate the "replacement" perspective and identify six interpretive fallacies that undermine its validity. These fallacies are: (1) equating token prediction with human intelligence, (2) treating LLMs as the average human, (3) interpreting alignment as explanation, (4) anthropomorphizing AI systems, (5) essentializing identities, and (6) substituting model data for human evidence. Each fallacy represents a potential misunderstanding about what LLMs are and what they can tell us about human cognition. The analysis distinguishes levels of similarity between LLMs and humans, particularly functional equivalence (outputs) versus mechanistic equivalence (processes), while highlighting both technical limitations (addressable through engineering) and conceptual limitations (arising from fundamental differences between statistical and biological intelligence). For each fallacy, specific safeguards are provided to guide responsible research practices. Ultimately, the analysis supports conceptualizing LLMs as pragmatic simulation tools--useful for role-play, rapid hypothesis testing, and computational modeling (provided their outputs are validated against human data)--rather than as replacements for human participants. This framework enables researchers to leverage language models productively while respecting the fundamental differences between machine intelligence and human thought.