Large Language Models as Psychological Simulators: A Methodological Guide
Not on the map. Why?
Abstract (arXiv)
Large language models (LLMs) offer emerging opportunities for psychological and behavioral research, but methodological guidance is lacking. This article provides a framework for using LLMs as psychological simulators across two primary applications: simulating roles and personas to explore diverse contexts, and serving as computational models to investigate cognitive processes. For simulation, we present methods for developing psychologically grounded personas that move beyond demographic categories, with strategies for validation against human data and use cases ranging from studying inaccessible populations to prototyping research instruments. For cognitive modeling, we synthesize emerging approaches for probing internal representations, methodological advances in causal interventions, and strategies for relating model behavior to human cognition. We address overarching challenges including prompt sensitivity, temporal limitations from training data cutoffs, and ethical considerations that extend beyond traditional human subjects review. Throughout, we emphasize the need for transparency about model capabilities and constraints. Together, this framework integrates emerging empirical evidence about LLM performance--including systematic biases, cultural limitations, and prompt brittleness--to help researchers wrangle these challenges and leverage the unique capabilities of LLMs in psychological research.
Key facts. Cited by 6 papers in the map (data through 14 Oct 2026). Builds on 42 of its 115 references that are in the map. First posted on arXiv 20 Jun 2025.
Builds on
42 of 115 references.
References · 42
- 2023Generative Agents: Interactive Simulacra of Human Behaviorcited by 3,568
- 2022OPT: Open Pre-trained Transformer Language Modelscited by 3,512
- 2022Locating and Editing Factual Associations in GPTcited by 2,691
- 2022Emergent Abilities of Large Language Modelscited by 2,512
- 2019BERT Rediscovers the Classical NLP Pipelinecited by 1,059
- 2021Probing Classifiers: Promises, Shortcomings, and Advancescited by 876
- 2023Whose Opinions Do Language Models Reflect?cited by 773
- 2019How multilingual is Multilingual BERT?cited by 645
- 2022Using cognitive psychology to understand GPT-3cited by 398
- 2023Role-Play with Large Language Modelscited by 294
- 2023Can Large Language Models Transform Computational Social Science?cited by 250
- 2024How to use and interpret activation patchingcited by 214
30 more
- 2024Two Tales of Persona in LLMs: A Survey of Role-Playing and Personalizationcited by 208
- 2023Do LLMs exhibit human-like response biases? A case study in survey designcited by 180
- 2023Cultural Bias and Cultural Alignment of Large Language Modelscited by 149
- 2024Large language models that replace human participants can harmfully misportray and flatten identity groupscited by 144
- 2023Turning large language models into cognitive modelscited by 67
- 2024Wisdom of the Silicon Crowd: LLM Ensemble Prediction Capabilities Rival Human Crowd Accuracycited by 67
- 2021First Align, then Predict: Understanding the Cross-Lingual Ability of Multilingual BERTcited by 65
- 2024Language Models Learn Rare Phenomena from Less Rare Phenomena: The Case of the Missing AANNscited by 65
- 2024Virtual Personas for Language Models via an Anthology of Backstoriescited by 51
- 2023Large language models predict human sensory judgments across six modalitiescited by 48
- 2023Diminished Diversity-of-Thought in a Standard Large Language Modelcited by 45
- 2023Language acquisition: do children and language models follow similar learning stages?cited by 41
- 2024Beyond Demographics: Aligning Role-playing LLM-based Agents Using Human Belief Networkscited by 39
- 2021The Grammar-Learning Trajectories of Neural Language Modelscited by 36
- 2024Large Language Models Portray Socially Subordinate Groups as More Homogeneous, Consistent with a Bias Observed in Humanscited by 31
- 2024Six Fallacies in Substituting Large Language Models for Human Participantscited by 24
- 2024Language Models as Models of Languagecited by 24
- 2024Do Language Models Exhibit Human-like Structural Priming Effects?cited by 21
- 2024Exploring the Frontiers of LLMs in Psychological Applications: A Comprehensive Reviewcited by 17
- 2024Data Authenticity, Consent, & Provenance for AI are all broken: what will it take to fix them?cited by 15
- 2023Techniques for supercharging academic writing with generative AIcited by 9
- 2024Large-scale moral machine experiment on large language modelscited by 9
- 2023Large language models can replicate cross-cultural differences in personalitycited by 8
- 2024Unveiling and Mitigating Bias in Mental Health Analysis with Large Language Modelscited by 5
- 2023Grounding Neuroscience in Behavioral Changes using Artificial Neural Networkscited by 4
- 2024Exploring Spatial Schema Intuitions in Large Language and Vision Modelscited by 4
- 2025When the LM misunderstood the human chuckled: Analyzing garden path effects in humans and language modelscited by 4
- 2024Surveying the Dead Minds: Historical-Psychological Text Analysis with Contextualized Construct Representation (CCR) for Classical Chinesecited by 3
- 2025The potential -- and the pitfalls -- of using pre-trained language models as cognitive science theoriescited by 2
- 2025Arti-"fickle" Intelligence: Using LLMs as a Tool for Inference in the Political and Social Sciencescited by 1
Built on by
Citing papers per year: 2025: 2; 2026: 4.
6 papers.
- 2025A validity-guided workflow for robust large language model research in psychologycited by 8
- 2025Large language models show fragile cognitive reasoning about human emotionscited by 2
- 2026Evaluating Cognitive Age Alignment in Interactive AI Agentscited by 2
- 2026Consequential Behaviour and Representational Fairness in the Validation of Synthetic Researchcited by 1
- 2026Six misconceptions about large language models: A minimal model and diagnostic taxonomycited by 0
- 2026Benchmarking large language model agent societies against human behavioural distributionscited by 0
- 2026Consequential Behaviour and Representational Fairness in the Validation of Synthetic Research23 Sep 2026
- 2026Benchmarking large language model agent societies against human behavioural distributions28 Aug 2026
- 2026Six misconceptions about large language models: A minimal model and diagnostic taxonomy19 Aug 2026
- 2026Evaluating Cognitive Age Alignment in Interactive AI Agents18 May 2026
- 2025Large language models show fragile cognitive reasoning about human emotions7 Aug 2025
- 2025A validity-guided workflow for robust large language model research in psychology6 Jul 2025