mlsanityData to 7 Oct 2026

Large language models that replace human participants can harmfully misportray and flatten identity groups

arXiv 2402.01908 ↗First posted 2 Feb 2024cs.CYcited by 137

Not on the map. Why?

Cited by 26 in LLM agents and tool use, but placed by its own abstract, which didn’t fit.

Abstract (arXiv)

Large language models (LLMs) are increasing in capability and popularity, propelling their application in new domains -- including as replacements for human participants in computational social science, user testing, annotation tasks, and more. In many settings, researchers seek to distribute their surveys to a sample of participants that are representative of the underlying human population of interest. This means in order to be a suitable replacement, LLMs will need to be able to capture the influence of positionality (i.e., relevance of social identities like gender and race). However, we show that there are two inherent limitations in the way current LLMs are trained that prevent this. We argue analytically for why LLMs are likely to both misportray and flatten the representations of demographic groups, then empirically show this on 4 LLMs through a series of human studies with 3200 participants across 16 demographic identities. We also discuss a third limitation about how identity prompts can essentialize identities. Throughout, we connect each limitation to a pernicious history of epistemic injustice against the value of lived experiences that explains why replacement is harmful for marginalized demographic groups. Overall, we urge caution in use cases where LLMs are intended to replace human participants whose identities are relevant to the task at hand. At the same time, in cases where the benefits of LLM replacement are determined to outweigh the harms (e.g., the goal is to supplement rather than fully replace, engaging human participants may cause them harm), we provide inference-time techniques that we empirically demonstrate do reduce, but do not remove, these harms.

Key facts. Cited by 137 papers in the map (data through 7 Oct 2026). First posted on arXiv 2 Feb 2024.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2024: 9; 2025: 51; 2026: 77.

137 papers.

Sort:

By area:

  1. 2024Quantifying the Persona Effect in LLM Simulationscited by 164
  2. 2024Survey of Cultural Awareness in Language Models: Text and Beyondcited by 101
  3. 2024Benchmarking Distributional Alignment of Large Language Modelscited by 58
  4. 2025SimBench: Benchmarking the Ability of Large Language Models to Simulate Human Behaviorscited by 48
  5. 2025Correlated Errors in Large Language Modelscited by 42
  6. 2025The Prompt Makes the Person(a): A Systematic Evaluation of Sociodemographic Persona Prompting for Large Language Modelscited by 40
  7. 2025Finetuning LLMs for Human Behavior Prediction in Social Science Experimentscited by 35
  8. 2025Beyond Demographics: Fine-tuning Large Language Models to Predict Individuals' Subjective Text Perceptionscited by 31
  9. 2025ChatBench: From Static Benchmarks to Human-AI Evaluationcited by 31
  10. 2025Value Profiles for Encoding Human Variationcited by 29
  11. 2024Towards interactive evaluations for interaction harms in human-AI systemscited by 27
  12. 2026This human study did not involve human subjects: Validating LLM simulations as behavioral evidencecited by 26
38 more
  1. 2025Multilingual Prompting for Improving LLM Generation Diversitycited by 25
  2. 2025Nuclear Deployed: Analyzing Catastrophic Risks in Decision-making of Autonomous LLM Agentscited by 23
  3. 2024Annotation alignment: Comparing LLM and human annotations of conversational safetycited by 22
  4. 2025Presumed Cultural Identity: How Names Shape LLM Responsescited by 21
  5. 2024Assessing Dialect Fairness and Robustness of Large Language Models in Reasoning Taskscited by 20
  6. 2025Valid Survey Simulations with Limited Human Data: The Roles of Prompting, Fine-Tuning, and Rectificationcited by 17
  7. 2025Cultural Learning-Based Culture Adaptation of Language Modelscited by 16
  8. 2024Real or Robotic? Assessing Whether LLMs Accurately Simulate Qualities of Human Responses in Dialoguecited by 15
  9. 2025DeepPersona: A Generative Engine for Scaling Deep Synthetic Personascited by 14
  10. 2025Debate-to-Detect: Reformulating Misinformation Detection as a Real-World Debate with Large Language Modelscited by 12
  11. 2025On the Alignment of Large Language Models with Global Human Opinioncited by 12
  12. 2025LLM-Based Social Simulations Require a Boundarycited by 11
  13. 2025From Prompts to Constructs: A Dual-Validity Framework for Large Language Model Research in Psychologycited by 9
  14. 2025Translation in the Hands of Many:Centering Lay Users in Machine Translation Interactionscited by 8
  15. 2025SusBench: An Online Benchmark for Evaluating Dark Pattern Susceptibility of Computer-Use Agentscited by 8
  16. 2025Optimizing Diversity and Quality through Base-Aligned Model Collaborationcited by 8
  17. 2025Rigor in AI: Doing Rigorous AI Work Requires a Broader, Responsible AI-Informed Conception of Rigorcited by 7
  18. 2025Moloch's Bargain: Emergent Misalignment When LLMs Compete for Audiencescited by 7
  19. 2025Poor Alignment and Steerability of Large Language Models: Evidence from College Admission Essayscited by 6
  20. 2025Large Language Models as Psychological Simulators: A Methodological Guidecited by 6
  21. 2025AIM-Bench: Evaluating Decision-making Biases of Agentic LLM as Inventory Managercited by 6
  22. 2024United in Diversity? Contextual Biases in LLM-Based Predictions of the 2024 European Parliament Electionscited by 5
  23. 2025AI Agent Behavioral Sciencecited by 5
  24. 2025DPRF: A Generalizable Dynamic Persona Refinement Framework for Optimizing Behavior Alignment Between Personalized LLM Role-Playing Agents and Humanscited by 5
  25. 2025Distribution Shift Alignment Helps LLMs Simulate Survey Response Distributionscited by 5
  26. 2026MatrAIx: Simulating the World with 8.3 Billion Persona Agentscited by 5
  27. 2025A large-scale evaluation of commonsense knowledge in humans and large language modelscited by 4
  28. 2025Measuring Lexical Diversity of Synthetic Data Generated through Fine-Grained Persona Promptingcited by 4
  29. 2025Words Like Knives: Backstory-Personalized Modeling and Detection of Violent Communicationcited by 4
  30. 2025ValueSim: Generating Backstories to Model Individual Value Systemscited by 4
  31. 2025German General Social Survey Personas: A Survey-Derived Persona Prompt Collection for Population-Aligned LLM Studiescited by 4
  32. 2026Identity, Cooperation and Framing Effects within Groups of Real and Simulated Humanscited by 4
  33. 2026Restoring Heterogeneity in LLM-based Social Simulation: An Audience Segmentation Approachcited by 4
  34. 2024The Life Cycle of Large Language Models: A Review of Biases in Educationcited by 3
  35. 2025Large Language Models are Near-Optimal Decision-Makers with a Non-Human Learning Behaviorcited by 3
  36. 2025How large language models judge and influence human cooperationcited by 3
  37. 2025Are LLM Agents Behaviorally Coherent? Latent Profiles for Social Simulationcited by 3
  38. 2025Investigating Political and Demographic Associations in Large Language Models Through Moral Foundations Theorycited by 3

50 most cited of 137 All 137 (JSON)