Emulate or Estimate? The Divergent Strengths of Base and Post-Trained Language Models for Opinion Simulation
Not on the map. Why?
Abstract (arXiv)
Large language models are increasingly used to simulate human opinions, but prior work reports conflicting results: some studies find promising alignment with human survey data, while others find persona collapse and weak demographic sensitivity. We propose that much of this conflict stems from conflating two distinct tasks. We call the first task emulation, in which models generate individual responses that aggregate into a population distribution. We call the second task estimation, in which models directly predict the population distribution. Evaluating six matched base and post-trained models on the Pew American Trends Panel, we find that base models are the stronger emulators: they produce response distributions closer to human ground truth and better preserve demographic structure. Post-trained models are generally the stronger estimators, producing more accurate distributional predictions when asked directly. We argue that model selection for human simulation should be guided by whether the task requires generating text or predicting distributions.
Key facts. No citing papers in the map yet (data through 7 Oct 2026). Builds on 18 of its 29 references that are in the map. First posted on arXiv 4 Aug 2026.
Builds on
18 of 29 references.
References · 18
- 2023Generative Agents: Interactive Simulacra of Human Behaviorcited by 3,500
- 2023Whose Opinions Do Language Models Reflect?cited by 759
- 2022Out of One, Many: Using Language Models to Simulate Human Samplescited by 737
- 2023Towards Measuring the Representation of Subjective Global Opinions in Language Modelscited by 424
- 2024LLM Agents Grounded in Self-Reports Enable General-Purpose Simulation of Individualscited by 235
- 2023Do LLMs exhibit human-like response biases? A case study in survey designcited by 176
- 2023Generative agent-based modeling with actions grounded in physical, social, or digital space using Concordiacited by 119
- 2025LLM Generated Persona is a Promise with a Catchcited by 77
- 2026The Assistant Axis: Situating and Stabilizing the Default Persona of Language Modelscited by 76
- 2025Language Model Fine-Tuning on Scaled Survey Data for Predicting Distributions of Public Opinionscited by 64
- 2025Position: Evaluating Generative AI Systems Is a Social Science Measurement Challengecited by 61
- 2024Benchmarking Distributional Alignment of Large Language Modelscited by 58
6 more
- 2025Specializing Large Language Models to Simulate Survey Response Distributions for Global Populationscited by 50
- 2024Virtual Personas for Language Models via an Anthology of Backstoriescited by 49
- 2024Random Silicon Sampling: Simulating Human Sub-Population Opinion Using a Large Language Model Based on Group-Level Demographic Informationcited by 48
- 2023Sociodemographic Prompting is Not Yet an Effective Approach for Simulating Subjective Judgments with LLMscited by 36
- 2026This human study did not involve human subjects: Validating LLM simulations as behavioral evidencecited by 26
- 2026Restoring Heterogeneity in LLM-based Social Simulation: An Audience Segmentation Approachcited by 4
Built on by
Not cited yet.