mlsanityData to 14 Oct 2026

The Limits of Simulated Societies: How Post-Training and Survey Fine-Tuning Erase Cross-Cultural Variance

arXiv 2609.25760 ↗First posted 22 Sep 2026cs.AIcited by 1

On the map: LLM alignment / RLHF / preference optimisation → Human values and AI feedback alignment

Abstract (arXiv)

Using large language models (LLMs) to simulate diverse human populations has the potential to transform many aspects of computational social science, yet many evaluations score the average response rather than the spread of opinion within real groups. Here, we develop a diagnostic framework that measures point accuracy alongside dispersion retention, the ratio of predicted to human standard deviation (dr), on 10,000 respondent--question pairs from the World Values Survey (WVS) spanning twelve countries and six continents. We evaluate eleven zero-shot language models and five variants fine-tuned on WVS data with SFT, DPO, and GRPO. We identify a failure mode we term textitconsensus collapse, where alignment training compresses outputs toward one stereotype per group. Along the post-training trajectory from the Llama~3.1 70B base to the Tulu~3 checkpoints, the first stage, supervised instruction tuning, removes half of the spread with minimal accuracy gain (dr 1.22 to 0.59; accuracy +0.9 points), the later stages do not restore it, and a gap opens between WEIRD and non-WEIRD countries that survey fine-tuning then deepens while pursuing higher point accuracy. The most accurate model (Tulu~3 70B-DPO fine-tuned on WVS, 57.9%) keeps half the human spread overall (dr = 0.50) and 11% of it for Nigeria, against 0.70--0.87 for WEIRD countries. Raising the sampling temperature to 1.0 leaves the Wasserstein-1 distance (wone) to human distributions unchanged for both fine-tuned DPO models, and GRPO on Qwen~3.5 9B does not restore the spread under either an accuracy reward or a distribution-shaped reward. Mixing the aligned model with an unaligned prior raises dr from 0.51 to 0.62 on a held-out split but leaves Nigeria at 0.36. Point accuracy alone therefore misjudges these simulators, and current post-training trades diversity for consensus.

Key facts. Cited by 1 paper in the map (data through 14 Oct 2026). Placed in Human values and AI feedback alignment, one of 10 directions in LLM alignment / RLHF / preference optimisation. Builds on 14 of its 23 references that are in the map. First posted on arXiv 22 Sep 2026.

Builds on

14 of 23 references.

References · 14

  1. 2022Training language models to follow instructions with human feedbackcited by 15,971
  2. 2024DeepSeekMath: Pushing the Limits of Mathematical Reasoning in Open Language Modelscited by 9,379
  3. 2023Direct Preference Optimization: Your Language Model is Secretly a Reward Modelcited by 8,065
  4. 2023QLoRA: Efficient Finetuning of Quantized LLMscited by 3,418
  5. 2024Tulu 3: Pushing Frontiers in Open Language Model Post-Trainingcited by 1,053
  6. 2023Whose Opinions Do Language Models Reflect?cited by 773
  7. 2022Out of One, Many: Using Language Models to Simulate Human Samplescited by 752
  8. 2023Towards Measuring the Representation of Subjective Global Opinions in Language Modelscited by 437
  9. 2024Large language models that replace human participants can harmfully misportray and flatten identity groupscited by 144
  10. 2024Random Silicon Sampling: Simulating Human Sub-Population Opinion Using a Large Language Model Based on Group-Level Demographic Informationcited by 49
  11. 2023AI-Augmented Surveys: Leveraging Large Language Models and Surveys for Opinion Predictioncited by 48
  12. 2025Finetuning LLMs for Human Behavior Prediction in Social Science Experimentscited by 39
2 more
  1. 2025Valid Survey Simulations with Limited Human Data: The Roles of Prompting, Fine-Tuning, and Rectificationcited by 18
  2. 2024Assessing Generalization for Subpopulation Representative Modeling via In-Context Learningcited by 4

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2026: 1.

1 paper.

Similar, not cited

Close in wording, no citation either way

  1. 2024Benchmarking Distributional Alignment of Large Language Models
  2. 2026Aligned but Flattened: Analyzing the Trade-off between Cultural Alignment and Diversity in LLMs
  3. 2024One fish, two fish, but not the whole sea: Alignment reduces language models' conceptual diversity
  4. 2026Scenario-based Probing and Steering Cultural Values in Large Language Models--Extended Version
  5. 2025SimBench: Benchmarking the Ability of Large Language Models to Simulate Human Behaviors
  6. 2025An Analysis of Large Language Models for Simulating User Responses in Surveys
  7. 2026Alignment by Stereotyping: How LLMs Sacrifice Individual Distinctiveness for Cultural Adaptation
  8. 2025The Alignment Veto: How Safety Training Suppresses Cultural Knowledge in LLMs