mlsanityData to 14 Oct 2026

LLM Social Simulations Are a Promising Research Method

arXiv 2504.02234 ↗First posted 3 Apr 2025cs.HC cs.AI cs.CLcited by 99

Not on the map. Why?

Cited by 30 in LLM agents and tool use, but placed by its own abstract, which didn’t fit.

Abstract (arXiv)

Accurate and verifiable large language model (LLM) simulations of human research subjects promise an accessible data source for understanding human behavior and training new AI systems. However, results to date have been limited, and few social scientists have adopted this method. In this position paper, we argue that the promise of LLM social simulations can be achieved by addressing five tractable challenges. We ground our argument in a review of empirical comparisons between LLMs and human research subjects, commentaries on the topic, and related work. We identify promising directions, including context-rich prompting and fine-tuning with social science datasets. We believe that LLM social simulations can already be used for pilot and exploratory studies, and more widespread use may soon be possible with rapidly advancing LLM capabilities. Researchers should prioritize developing conceptual models and iterative evaluations to make the best use of new AI systems.

Key facts. Cited by 99 papers in the map (data through 14 Oct 2026). Builds on 86 of its 193 references that are in the map. First posted on arXiv 3 Apr 2025.

Builds on

86 of 193 references.

References · 86

  1. 2017Attention Is All You Needcited by 60,361
  2. 2022Chain-of-Thought Prompting Elicits Reasoning in Large Language Modelscited by 12,459
  3. 2017On Calibration of Modern Neural Networkscited by 4,960
  4. 2023Generative Agents: Interactive Simulacra of Human Behaviorcited by 3,568
  5. 2023QLoRA: Efficient Finetuning of Quantized LLMscited by 3,418
  6. 2022Constitutional AI: Harmlessness from AI Feedbackcited by 3,050
  7. 2019The Curious Case of Neural Text Degenerationcited by 2,916
  8. 2023Sparks of Artificial General Intelligence: Early experiments with GPT-4cited by 2,592
  9. 2017Interpretability Beyond Feature Attribution: Quantitative Testing with Concept Activation Vectors (TCAV)cited by 1,319
  10. 2018Hierarchical Neural Story Generationcited by 1,193
  11. 2023Fine-tuning Aligned Language Models Compromises Safety, Even When Users Do Not Intend To!cited by 1,154
  12. 2023Steering Llama 2 via Contrastive Activation Additioncited by 1,085
38 more
  1. 2023Towards Understanding Sycophancy in Language Modelscited by 1,040
  2. 2020Language (Technology) is Power: A Critical Survey of "Bias" in NLPcited by 956
  3. 2023Progress measures for grokking via mechanistic interpretabilitycited by 839
  4. 2023Whose Opinions Do Language Models Reflect?cited by 773
  5. 2022Out of One, Many: Using Language Models to Simulate Human Samplescited by 752
  6. 2023The Linear Representation Hypothesis and the Geometry of Large Language Modelscited by 716
  7. 2016Fair prediction with disparate impact: A study of bias in recidivism prediction instrumentscited by 672
  8. 2022Using Large Language Models to Simulate Multiple Humans and Replicate Human Subject Studiescited by 515
  9. 2023Towards Measuring the Representation of Subjective Global Opinions in Language Modelscited by 437
  10. 2021Towards Out-Of-Distribution Generalization: A Surveycited by 436
  11. 2024Mechanistic Interpretability for AI Safety -- A Reviewcited by 422
  12. 2023SOTOPIA: Interactive Evaluation for Social Intelligence in Language Agentscited by 367
  13. 2024Alignment faking in large language modelscited by 360
  14. 2023Role-Play with Large Language Modelscited by 294
  15. 2019Lipstick on a Pig: Debiasing Methods Cover up Systematic Gender Biases in Word Embeddings But do not Remove Themcited by 268
  16. 2024LLM Agents Grounded in Self-Reports Enable General-Purpose Simulation of Individualscited by 249
  17. 2023PersonaLLM: Investigating the Ability of Large Language Models to Express Personality Traitscited by 234
  18. 2023Evaluating Large Language Models in Theory of Mind Taskscited by 222
  19. 2024From Persona to Personalization: A Survey on Role-Playing Language Agentscited by 212
  20. 2021Towards Causal Representation Learningcited by 209
  21. 2022Language models show human-like content effects on reasoning taskscited by 173
  22. 2016Linear Algebraic Structure of Word Senses, with Applications to Polysemycited by 165
  23. 2023Sociotechnical Safety Evaluation of Generative AI Systemscited by 157
  24. 2024Analyzing the Generalization and Reliability of Steering Vectorscited by 143
  25. 2018Invariance, Causality and Robustnesscited by 123
  26. 2023Mechanistically analyzing the effects of fine-tuning on procedurally defined taskscited by 115
  27. 2024Mind Your Step (by Step): Chain-of-Thought can Reduce Performance on Tasks where Thinking Makes Humans Worsecited by 109
  28. 2023Consciousness in Artificial Intelligence: Insights from the Science of Consciousnesscited by 104
  29. 2023Artificial Artificial Artificial Intelligence: Crowd Workers Widely Use Large Language Models for Text Production Taskscited by 103
  30. 2024Sycophancy in Large Language Models: Causes and Mitigationscited by 101
  31. 2024The Butterfly Effect of Altering Prompts: How Small Changes and Jailbreaks Affect Large Language Model Performancecited by 100
  32. 2024Systematic Biases in LLM Simulations of Debatescited by 99
  33. 2023CoMPosT: Characterizing and Evaluating Caricature in LLM Simulationscited by 97
  34. 2023Levels of AGI for Operationalizing Progress on the Path to AGIcited by 96
  35. 2023Aligning Large Language Models through Synthetic Feedbackcited by 75
  36. 2023Questioning the Survey Responses of Large Language Modelscited by 74
  37. 2024Hidden Persuaders: LLMs' Political Leaning and Their Influence on Voterscited by 71
  38. 2025Language Models Use Trigonometry to Do Additioncited by 70

50 most cited of 86 All 86 (JSON)

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2025: 52; 2026: 47.

99 papers.

Sort:

By area:

  1. 2025SocioVerse: A World Model for Social Simulation Powered by LLM Agents and A Pool of 10 Million Real-World Userscited by 55
  2. 2025SimBench: Benchmarking the Ability of Large Language Models to Simulate Human Behaviorscited by 52
  3. 2026HumanLM: Simulating Users with State Alignment Beats Response Imitationcited by 37
  4. 2026This human study did not involve human subjects: Validating LLM simulations as behavioral evidencecited by 29
  5. 2025RecoWorld: Building Simulated Environments for Agentic Recommender Systemscited by 19
  6. 2025Valid Survey Simulations with Limited Human Data: The Roles of Prompting, Fine-Tuning, and Rectificationcited by 18
  7. 2025ReviewerToo: Should AI Join The Program Committee? A Look At The Future of Peer Reviewcited by 16
  8. 2025The threat of analytic flexibility in using large language models to simulate human datacited by 15
  9. 2025The PIMMUR Principles: Ensuring Validity in Collective Behavior of LLM Societiescited by 14
  10. 2026Exploring Silicon-Based Societies: An Early Study of the Moltbook Agent Communitycited by 13
  11. 2025LLM-Based Social Simulations Require a Boundarycited by 12
  12. 2025From the Fluency Fallacy to the Micro-to-Macro Validity Gap: Opportunities and Pitfalls of LLMs in Social Simulationcited by 10
38 more
  1. 2025Breaking Bad Tokens: Detoxification of LLMs Using Sparse Autoencoderscited by 9
  2. 2025Many LLMs Are More Utilitarian Than Onecited by 8
  3. 2025Multi-Actor Generative Artificial Intelligence as a Game Enginecited by 8
  4. 2025CoBRA: Programming Cognitive Bias in Social Agents Using Classic Social Science Experimentscited by 8
  5. 2025Moloch's Bargain: Emergent Misalignment When LLMs Compete for Audiencescited by 8
  6. 2025Aligning LLM with human travel choices: a persona-based embedding learning approachcited by 7
  7. 2025Valid Inference with Imperfect Synthetic Datacited by 7
  8. 2025Large Language Models as Virtual Survey Respondents: Evaluating Sociodemographic Response Generationcited by 7
  9. 2026One Model, All Roles: Multi-Turn, Multi-Agent Self-Play Reinforcement Learning for Conversational Social Intelligencecited by 7
  10. 2026Diversity Collapse in Multi-Agent LLM Systems: Structural Coupling and Collective Failure in Open-Ended Idea Generationcited by 7
  11. 2025Beyond Brainstorming: What Drives High-Quality Scientific Ideas? Lessons from Multi-Agent Collaborationcited by 6
  12. 2025Computational Turing Test Reveals Systematic Differences Between Human and AI Languagecited by 6
  13. 2026Generative AI collective behavior needs an interactionist paradigmcited by 6
  14. 2026Persona Generators: Generating Diverse Synthetic Personas for Arbitrary Contextscited by 6
  15. 2026One Frozen Simulator Is Not Enough: Simulator Collapse in Multi-Agent RLcited by 6
  16. 2025Seeing Through Deception: Uncovering Misleading Creator Intent in Multimodal News with Vision-Language Modelscited by 5
  17. 2025See, Think, Act: Online Shopper Behavior Simulation with VLM Agentscited by 5
  18. 2026Overstating Attitudes, Ignoring Networks: LLM Biases in Simulating Misinformation Susceptibilitycited by 5
  19. 2025A large-scale evaluation of commonsense knowledge in humans and large language modelscited by 4
  20. 2025Noise, Adaptation, and Strategy: Assessing LLM Fidelity in Decision-Makingcited by 4
  21. 2025Are LLM Agents Behaviorally Coherent? Latent Profiles for Social Simulationcited by 4
  22. 2025Computational Basis of LLM's Decision Making in Social Simulationcited by 3
  23. 2025Integrating Counterfactual Simulations with Language Models for Explaining Multi-Agent Behaviourcited by 3
  24. 2025Scaling Law in LLM Simulated Personality: More Detailed and Realistic Persona Profile Is All You Needcited by 3
  25. 2025Simulating and Experimenting with Social Media Mobilization Using LLM Agentscited by 3
  26. 2025Agent-Kernel: A MicroKernel Multi-Agent System Framework for Adaptive Social Simulation Powered by LLMscited by 3
  27. 2025Agent-based simulation of online social networks and disinformationcited by 3
  28. 2026Evaluating Cooperation in LLM Social Groups through Elected Leadershipcited by 3
  29. 2026The Judgment-Consequence Gap: LLM Moral Reasoning in Healthcare Decisionscited by 3
  30. 2025A closer look at how large language models trust humans: patterns and biasescited by 2
  31. 2025Infected Smallville: How Disease Threat Shapes Sociality in LLM Agentscited by 2
  32. 2025Large language models show fragile cognitive reasoning about human emotionscited by 2
  33. 2025Towards Operational Validation of LLM-Agent Social Simulations: A Replicated Study of a Reddit-like Technology Forumcited by 2
  34. 2025Tailored untruths: How personalisation challenges LLM safeguardscited by 2
  35. 2025Generative AI in Sociological Research: State of the Disciplinecited by 2
  36. 2025Can Finetuing LLMs on Small Human Samples Increase Heterogeneity, Alignment, and Belief-Action Coherence?cited by 2
  37. 2025Multilingual Agent-Based World Modeling for Social Sciencecited by 2
  38. 2026Sentipolis: Emotion-Aware Agents for Social Simulationscited by 2

50 most cited of 99 All 99 (JSON)