Evaluating the Use of Large Language Models as Synthetic Social Agents in Social Science Research
On the map: LLM agents and tool use → Social simulation and multi-agent debate
Abstract (arXiv)
Large Language Models (LLMs) are being increasingly used as synthetic agents in social science, in applications ranging from augmenting survey responses to powering multi-agent simulations. This paper outlines cautions that should be taken when interpreting LLM outputs and proposes a pragmatic reframing for the social sciences in which LLMs are used as high-capacity pattern matchers for quasi-predictive interpolation under explicit scope conditions and not as substitutes for probabilistic inference. Practical guardrails such as independent draws, preregistered human baselines, reliability-aware validation, and subgroup calibration, are introduced so that researchers may engage in useful prototyping and forecasting while avoiding category errors.
Key facts. Cited by 2 papers in the map (data through 14 Oct 2026). Placed in Social simulation and multi-agent debate, one of 16 directions in LLM agents and tool use. Builds on 21 of its 32 references that are in the map. First posted on arXiv 30 Sep 2025.
Builds on
21 of 32 references.
Closest ancestors · 2
Same direction, newest first
- 2024LLM Agents Grounded in Self-Reports Enable General-Purpose Simulation of Individuals15 Nov
- 2023Generative Agents: Interactive Simulacra of Human Behavior7 Apr
Other references · 19
- 2017Attention Is All You Needcited by 60,361
- 2020Language Models are Few-Shot Learnerscited by 32,419
- 2023Judging LLM-as-a-Judge with MT-Bench and Chatbot Arenacited by 8,626
- 2020Scaling Laws for Neural Language Modelscited by 6,586
- 2021On the Opportunities and Risks of Foundation Modelscited by 4,033
- 2022Emergent Abilities of Large Language Modelscited by 2,512
- 2021An Explanation of In-context Learning as Implicit Bayesian Inferencecited by 938
- 2023Whose Opinions Do Language Models Reflect?cited by 773
- 2022Out of One, Many: Using Language Models to Simulate Human Samplescited by 752
- 2022Using Large Language Models to Simulate Multiple Humans and Replicate Human Subject Studiescited by 515
- 2024From Crowdsourced Data to High-Quality Benchmarks: Arena-Hard and BenchBuilder Pipelinecited by 500
- 2024Humans or LLMs as the Judge? A Study on Judgement Biasescited by 323
7 more
- 2024Judging the Judges: A Systematic Study of Position Bias in LLM-as-a-Judgecited by 269
- 2024Self-Preference Bias in LLM-as-a-Judgecited by 258
- 2024In-Context Language Learning: Architectures and Algorithmscited by 89
- 2023In-Context Learning through the Bayesian Prismcited by 83
- 2023Turning large language models into cognitive modelscited by 67
- 2024Is In-Context Learning in Large Language Models Bayesian? A Martingale Perspectivecited by 57
- 2025Predicting Field Experiments with Large Language Modelscited by 2
Built on by
Citing papers per year: 2026: 2.
2 papers.
Similar, not cited
Close in wording, no citation either way
- 2025Beyond Static Responses: Multi-Agent LLM Systems as a New Paradigm for Social Science Research
- 2025Exploring Large Language Model Agents for Piloting Social Experiments
- 2024Sense and Sensitivity: Evaluating the simulation of social dynamics via Large Language Models
- 2025Do Large Language Models Solve the Problems of Agent-Based Modeling? A Critical Review of Generative Social Simulations
- 2024LLM-Augmented Agent-Based Modelling for Social Simulations: Challenges and Opportunities
- 2026Mapping the Emerging Social Science of Large Language Models
- 2025Leveraging LLM-based agents for social science research: insights from citation network simulations
- 2025Are LLM Agents Behaviorally Coherent? Latent Profiles for Social Simulation