mlsanityData to 14 Oct 2026

Specializing Large Language Models to Simulate Survey Response Distributions for Global Populations

arXiv 2502.07068 ↗First posted 10 Feb 2025cs.CLcited by 53

Not on the map. Why?

Abstract (arXiv)

Large-scale surveys are essential tools for informing social science research and policy, but running surveys is costly and time-intensive. If we could accurately simulate group-level survey results, this would therefore be very valuable to social science research. Prior work has explored the use of large language models (LLMs) for simulating human behaviors, mostly through prompting. In this paper, we are the first to specialize LLMs for the task of simulating survey response distributions. As a testbed, we use country-level results from two global cultural surveys. We devise a fine-tuning method based on first-token probabilities to minimize divergence between predicted and actual response distributions for a given question. Then, we show that this method substantially outperforms other methods and zero-shot classifiers, even on unseen questions, countries, and a completely unseen survey. While even our best models struggle with the task, especially on unseen questions, our results demonstrate the benefits of specialization for simulation, which may accelerate progress towards sufficiently accurate simulation in the future.

Key facts. Cited by 53 papers in the map (data through 14 Oct 2026). Builds on 16 of its 30 references that are in the map. First posted on arXiv 10 Feb 2025.

Builds on

16 of 30 references.

References · 16

  1. 2023LLaMA: Open and Efficient Foundation Language Modelscited by 14,132
  2. 2022Out of One, Many: Using Language Models to Simulate Human Samplescited by 752
  3. 2022Using Large Language Models to Simulate Multiple Humans and Replicate Human Subject Studiescited by 515
  4. 2023Towards Measuring the Representation of Subjective Global Opinions in Language Modelscited by 437
  5. 2023Assessing Cross-Cultural Alignment between ChatGPT and Human Societies: An Empirical Studycited by 206
  6. 2022Probing Pre-Trained Language Models for Cross-Cultural Differences in Valuescited by 149
  7. 2024Investigating Cultural Alignment of Large Language Modelscited by 143
  8. 2022Stop Measuring Calibration When Humans Disagreecited by 71
  9. 2024Can multiple-choice questions really be useful in detecting the abilities of LLMs?cited by 69
  10. 2024Random Silicon Sampling: Simulating Human Sub-Population Opinion Using a Large Language Model Based on Group-Level Demographic Informationcited by 49
  11. 2024Evaluating Cultural Adaptability of a Large Language Model via Simulation of Synthetic Personascited by 37
  12. 2023Assessing LLMs for Moral Value Pluralismcited by 34
4 more
  1. 2023Investigating Uncertainty Calibration of Aligned Language Models under the Multiple-Choice Settingcited by 20
  2. 2024Interpreting Predictive Probabilities: Model Confidence or Human Label Variation?cited by 15
  3. 2023On the Calibration of Multilingual Question Answering LLMscited by 6
  4. 2024Revealing Fine-Grained Values and Opinions in Large Language Modelscited by 5

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2025: 21; 2026: 32.

53 papers.

Sort:

By area:

  1. 2025Language Model Fine-Tuning on Scaled Survey Data for Predicting Distributions of Public Opinionscited by 66
  2. 2025Beyond Demographics: Fine-tuning Large Language Models to Predict Individuals' Subjective Text Perceptionscited by 33
  3. 2025Valid Survey Simulations with Limited Human Data: The Roles of Prompting, Fine-Tuning, and Rectificationcited by 18
  4. 2025The threat of analytic flexibility in using large language models to simulate human datacited by 15
  5. 2025LLM-Based Social Simulations Require a Boundarycited by 12
  6. 2026Assessing the Reliability of Persona-Conditioned LLMs as Synthetic Survey Respondentscited by 10
  7. 2025Revisiting LLM Value Probing Strategies: Are They Robust and Expressive?cited by 8
  8. 2025Flipping Against All Odds: Reducing LLM Coin Flip Bias via Verbalized Rejection Samplingcited by 7
  9. 2025Large Language Models as Virtual Survey Respondents: Evaluating Sociodemographic Response Generationcited by 7
  10. 2025Zhyper: Factorized Hypernetworks for Conditioned LLM Fine-Tuningcited by 5
  11. 2025Distribution Shift Alignment Helps LLMs Simulate Survey Response Distributionscited by 5
  12. 2025German General Social Survey Personas: A Survey-Derived Persona Prompt Collection for Population-Aligned LLM Studiescited by 5
38 more
  1. 2026When Synthetic Users Fail: A Cross-Domain Benchmark of LLM-Simulated Human Survey Responsescited by 5
  2. 2026LLMs as Strategic Actors: Behavioral Alignment, Risk Calibration, and Argumentation Framing in Geopolitical Simulationscited by 4
  3. 2026When Can Digital Personas Reliably Approximate Human Survey Findings?cited by 4
  4. 2026Beyond the Mean: Three-Axis Fidelity for Aligning LLM-Based Survey Simulators from Small Pilot Datacited by 3
  5. 2025From Word to World: Evaluate and Mitigate Culture Bias in LLMs via Word Association Testcited by 2
  6. 2025Improving the Distributional Alignment of LLMs using Supervisioncited by 2
  7. 2025Prompts to Proxies: Emulating Human Preferences via a Compact LLM Ensemblecited by 2
  8. 2025Predicting Effects, Missing Distributions: Evaluating LLMs as Human Behavior Simulators in Operations Managementcited by 2
  9. 2025Tailored untruths: How personalisation challenges LLM safeguardscited by 2
  10. 2025Model-Free Assessment of Simulator Fidelity via Quantile Curvescited by 2
  11. 2026SYN-DIGITS: A Synthetic Control Framework for Calibrated Digital Twin Simulationcited by 2
  12. 2026Preregistration for Experiments with AI Agentscited by 2
  13. 2025Too Open for Opinion? Embracing Open-Endedness in Large Language Models for Social Simulationcited by 1
  14. 2025Graph-Based Alternatives to LLMs for Human Simulationcited by 1
  15. 2026RADIUS: Ranking, Distribution, and Significance - A Comprehensive Alignment Suite for Survey Simulationcited by 1
  16. 2026Statistical realism is not evidence that LLMs can estimate treatment effects in social science experimentscited by 1
  17. 2026When Seekers Are Hard to Help: Evaluating Emotional Support Dialogue Systems in Worst-Case Interactionscited by 1
  18. 2026In-Context Learning for the Imputation of Public Opinion Data with Large Language Modelscited by 1
  19. 2026Silicon Sampling via Cross-Survey Transfercited by 1
  20. 2026When Can LLM Digital Twins Reduce Human Measurement? From Behavioral Fidelity to Statistical Substitutabilitycited by 1
  21. 2026When Can You Trust Your Synthetic Users? Diagnostics and Corrections for LLM Consumer Panelscited by 1
  22. 2025Unsupervised Evaluation of Multi-Turn Objective-Driven Interactionscited by 0
  23. 2025Cross-cultural value alignment frameworks for responsible AI governance: Evidence from China-West comparative analysiscited by 0
  24. 2026MAPLE: Metadata Conditioned LLM Pretraining for Locale-Aware Question Answeringcited by 0
  25. 2026Beyond Static Snapshots: Dynamic Modeling and Forecasting of Group-Level Value Evolution with Large Language Modelscited by 0
  26. 2026Evidence-based Distributional Alignment for Large Language Modelscited by 0
  27. 2026Adaptive Querying with AI Persona Priorscited by 0
  28. 2026Validated Hypotheses as a Lens for Human-Likeness Evaluation in AI Agentscited by 0
  29. 2026Two Wrongs, No Right: Auditing Social-Desirability Bias in LLM Annotators for Computational Social Sciencecited by 0
  30. 2026LLMs Infer Cultural Context but Fail to Apply It When Respondingcited by 0
  31. 2026Humans Are More Diverse: Frontier LLMs Show Extreme Policies in Idealised AI Development Racescited by 0
  32. 2026Emulate or Estimate? The Divergent Strengths of Base and Post-Trained Language Models for Opinion Simulationcited by 0
  33. 2026Distributional Validity of a Korean Synthetic Persona Panel: Evidence From the Korea Media Panel Surveycited by 0
  34. 2026CultureMINE: Datasets and Methods for Improving the Cultural Capabilities of NLP Systemscited by 0
  35. 2026Understanding Reliability in LLM-based Human Behavior Simulationcited by 0
  36. 2026Cultural Divergence Preservation: Diagnosing Flattening and Caricature in LLM-Simulated Survey Populationscited by 0
  37. 2026Simulating Respondents, Not Single Questions: Coherent Survey Generation with Large Language Modelscited by 0
  38. 2026Population Fidelity: Evaluating Population Representativeness in LLMscited by 0

50 most cited of 53 All 53 (JSON)