Alignment by Stereotyping: How LLMs Sacrifice Individual Distinctiveness for Cultural Adaptation
Not on the map. Why?
Abstract (arXiv)
Large language models are increasingly deployed for personalized interaction, and demographic conditioning via user profiles is a widely adopted strategy for cultural adaptation. We ask whether this approach genuinely serves individual users or achieves accuracy by erasing individual distinctiveness. Studying seven models including frontier GPT-5.1 on the World Values Survey, we find that demographic profiles improve value alignment accuracy for most models, but at a systematic cost to individuality. That is, models pull responses toward demographic group centroids rather than preserving individual differences, a behavioral pattern we term alignment by stereotyping. Permutation tests (10,000 permutations, six demographic attributes, seven models) certify that top-performing models compress individuals far above the human baseline; within-family scaling amplifies this tradeoff while degrading intrinsic cultural understanding. Using a synthetic dialogue dataset validated on real human-chatbot conversations from PRISM (Kirk et al., 2024), we further show that distributing demographic signals across conversational turns partially suppresses prototype retrieval compared to compact demographic labels, a finding validated on real conversations via PRISM but requiring replication at larger scale.
Key facts. No citing papers in the map yet (data through 14 Oct 2026). Builds on 16 of its 25 references that are in the map. First posted on arXiv 5 Sep 2026.
Builds on
16 of 25 references.
References · 16
- 2023Towards Measuring the Representation of Subjective Global Opinions in Language Modelscited by 437
- 2022Probing Pre-Trained Language Models for Cross-Cultural Differences in Valuescited by 149
- 2023Cultural Alignment in Large Language Models: An Explanatory Analysis Based on Hofstede's Cultural Dimensionscited by 112
- 2023Sensitivity, Performance, Robustness: Deconstructing the Effect of Sociodemographic Promptingcited by 81
- 2024WorldValuesBench: A Large-Scale Benchmark Dataset for Multi-Cultural Value Awareness of Language Modelscited by 60
- 2024PAD: Personalized Alignment of LLMs at Decoding-Timecited by 60
- 2025Randomness, Not Representation: The Unreliability of Evaluating Cultural Alignment in LLMscited by 54
- 2024Can Language Models Reason about Individualistic Human Values and Preferences?cited by 34
- 2025An Evaluation of Cultural Value Alignment in LLMcited by 28
- 2025PROPER: A Progressive Learning Framework for Personalized Large Language Models with Group-Level Adaptationcited by 24
- 2025Teaching Language Models to Evolve with Users: Dynamic Profile Modeling for Personalized Alignmentcited by 24
- 2025Reading Between the Prompts: How Stereotypes Shape LLM's Implicit Personalizationcited by 24
4 more
- 2024The Impossibility of Fair LLMscited by 22
- 2024Cultural Conditioning or Placebo? On the Effectiveness of Socio-Demographic Promptingcited by 22
- 2024Fairness Testing of Large Language Models in Role-Playingcited by 11
- 2025Do language models accommodate their users? A study of linguistic convergencecited by 7
Built on by
Not cited yet.
Similar, not cited
Close in wording, no citation either way
- 2026Aligned but Flattened: Analyzing the Trade-off between Cultural Alignment and Diversity in LLMs
- 2025DiverValue-Bench: A Benchmark and Fine-Tuning Framework for Aligning Large Language Models with Diverse Human Values
- 2024One fish, two fish, but not the whole sea: Alignment reduces language models' conceptual diversity
- 2023From Instructions to Intrinsic Human Values -- A Survey of Alignment Goals for Big Models
- 2025Operationalizing Pluralistic Values in Large Language Model Alignment Reveals Trade-offs in Safety, Inclusivity, and Model Behavior
- 2026The Limits of Simulated Societies: How Post-Training and Survey Fine-Tuning Erase Cross-Cultural Variance
- 2026CommunityBench: Benchmarking Community-Level Alignment across Diverse Groups and Tasks
- 2026Can Persona-Prompted LLMs Emulate Subgroup Values? An Empirical Analysis of Generalisability and Fairness in Cultural Alignment