mlsanityData to 7 Oct 2026

Out of One, Many: Using Language Models to Simulate Human Samples

arXiv 2209.06899 ↗First posted 14 Sep 2022cs.LG cs.CLcited by 737

Not on the map. Why?

Cited by 140 in LLM agents and tool use, but placed by its own abstract, which didn’t fit.

Abstract (arXiv)

We propose and explore the possibility that language models can be studied as effective proxies for specific human sub-populations in social science research. Practical and research applications of artificial intelligence tools have sometimes been limited by problematic biases (such as racism or sexism), which are often treated as uniform properties of the models. We show that the "algorithmic bias" within one such tool -- the GPT-3 language model -- is instead both fine-grained and demographically correlated, meaning that proper conditioning will cause it to accurately emulate response distributions from a wide variety of human subgroups. We term this property "algorithmic fidelity" and explore its extent in GPT-3. We create "silicon samples" by conditioning the model on thousands of socio-demographic backstories from real human participants in multiple large surveys conducted in the United States. We then compare the silicon and human samples to demonstrate that the information contained in GPT-3 goes far beyond surface similarity. It is nuanced, multifaceted, and reflects the complex interplay between ideas, attitudes, and socio-cultural context that characterize human attitudes. We suggest that language models with sufficient algorithmic fidelity thus constitute a novel and powerful tool to advance understanding of humans and society across a variety of disciplines.

Key facts. Cited by 737 papers in the map (data through 7 Oct 2026). Builds on 7 of its 44 references that are in the map. First posted on arXiv 14 Sep 2022.

Builds on

7 of 44 references.

References · 7

  1. 2020Language Models are Few-Shot Learnerscited by 32,199
  2. 2019Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformercited by 14,491
  3. 2019Transformer-XL: Attentive Language Models Beyond a Fixed-Length Contextcited by 1,830
  4. 2016Semantics derived automatically from language corpora contain human-like biasescited by 894
  5. 2020Towards a Human-like Open-Domain Chatbotcited by 467
  6. 2017Word Embeddings Quantify 100 Years of Gender and Ethnic Stereotypescited by 251
  7. 2020The Next Decade in AI: Four Steps Towards Robust Artificial Intelligencecited by 194

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2022: 2; 2023: 102; 2024: 146; 2025: 197; 2026: 290.

737 papers.

Sort:

By area:

  1. 2023Generative Agents: Interactive Simulacra of Human Behaviorcited by 3,500
  2. 2023A Survey on Large Language Model based Autonomous Agentscited by 2,203
  3. 2023ChatGPT Outperforms Crowd-Workers for Text-Annotation Taskscited by 849
  4. 2023Whose Opinions Do Language Models Reflect?cited by 759
  5. 2023AlpacaFarm: A Simulation Framework for Methods that Learn from Human Feedbackcited by 738
  6. 2023From Pretraining Data to Language Models to Downstream Tasks: Tracking the Trails of Political Biases Leading to Unfair NLP Modelscited by 257
  7. 2023Large Language Models Empowered Agent-based Modeling and Simulation: A Survey and Perspectivescited by 253
  8. 2023Challenges and Applications of Large Language Modelscited by 251
  9. 2023Prompting Large Language Model for Machine Translation: A Case Studycited by 247
  10. 2023Can Large Language Models Transform Computational Social Science?cited by 245
  11. 2023Evaluating the Moral Beliefs Encoded in LLMscited by 211
  12. 2023Bias Runs Deep: Implicit Reasoning Biases in Persona-Assigned LLMscited by 191
38 more
  1. 2025AgentSociety: Large-Scale Simulation of LLM-Driven Generative Agents Advances Understanding of Human Behaviors and Societycited by 180
  2. 2023Do LLMs exhibit human-like response biases? A case study in survey designcited by 176
  3. 2024A Roadmap to Pluralistic Alignmentcited by 175
  4. 2023GPTs are GPTs: An Early Look at the Labor Market Impact Potential of Large Language Modelscited by 174
  5. 2024Quantifying the Persona Effect in LLM Simulationscited by 164
  6. 2023In-Context Impersonation Reveals Large Language Models' Strengths and Biasescited by 154
  7. 2024Exploring Large Language Model based Intelligent Agents: Definitions, Methods, and Prospectscited by 130
  8. 2023Generative agent-based modeling with actions grounded in physical, social, or digital space using Concordiacited by 119
  9. 2023Generative Language Models Exhibit Social Identity Biasescited by 107
  10. 2024Best Practices and Lessons Learned on Synthetic Datacited by 104
  11. 2023Artificial Artificial Artificial Intelligence: Crowd Workers Widely Use Large Language Models for Text Production Taskscited by 102
  12. 2024Large Language Models as Urban Residents: An LLM Agent Framework for Personal Mobility Generationcited by 99
  13. 2024Systematic Biases in LLM Simulations of Debatescited by 98
  14. 2023ChatGPT-4 Outperforms Experts and Crowd Workers in Annotating Political Twitter Messages with Zero-Shot Learningcited by 97
  15. 2023Value Kaleidoscope: Engaging AI with Pluralistic Human Values, Rights, and Dutiescited by 97
  16. 2025LLM Social Simulations Are a Promising Research Methodcited by 97
  17. 2023CoMPosT: Characterizing and Evaluating Caricature in LLM Simulationscited by 94
  18. 2023Language Model Behavior: A Comprehensive Surveycited by 92
  19. 2023Can Large Language Models Serve as Rational Players in Game Theory? A Systematic Analysiscited by 91
  20. 2024Measuring Political Bias in Large Language Models: What Is Said and How It Is Saidcited by 89
  21. 2023Mitigating Label Biases for In-context Learningcited by 88
  22. 2023Personalisation within bounds: A risk taxonomy and policy framework for the alignment of large language models with personalised feedbackcited by 86
  23. 2023The Participatory Turn in AI Design: Theoretical Foundations and the Current State of Practicecited by 86
  24. 2023MetaAgents: Large Language Model Based Agents for Decision-Making on Teamingcited by 85
  25. 2023Sensitivity, Performance, Robustness: Deconstructing the Effect of Sociodemographic Promptingcited by 80
  26. 2024Evaluating Large Language Model Biases in Persona-Steered Generationcited by 78
  27. 2023Training Socially Aligned Language Models on Simulated Social Interactionscited by 77
  28. 2025LLM Generated Persona is a Promise with a Catchcited by 77
  29. 2024From Individual to Society: A Survey on Social Simulation Driven by Large Language Model-based Agentscited by 76
  30. 2023EvalLM: Interactive Evaluation of Large Language Model Prompts on User-Defined Criteriacited by 73
  31. 2022Moral Mimicry: Large Language Models Produce Moral Rationalizations Tailored to Political Identitycited by 71
  32. 2023Questioning the Survey Responses of Large Language Modelscited by 70
  33. 2023CompeteAI: Understanding the Competition Dynamics in Large Language Model-based Agentscited by 65
  34. 2023Ignore This Title and HackAPrompt: Exposing Systemic Vulnerabilities of LLMs through a Global Scale Prompt Hacking Competitioncited by 65
  35. 2023Simulating Social Media Using Large Language Models to Evaluate Alternative News Feed Algorithmscited by 64
  36. 2025Language Model Fine-Tuning on Scaled Survey Data for Predicting Distributions of Public Opinionscited by 64
  37. 2024Can Large Language Model Agents Simulate Human Trust Behavior?cited by 62
  38. 2024Nicer Than Humans: How do Large Language Models Behave in the Prisoner's Dilemma?cited by 62

50 most cited of 737 All 737 (JSON)