# Randomness, Not Representation: The Unreliability of Evaluating Cultural Alignment in LLMs

arXiv: https://arxiv.org/abs/2503.08688 · HTML: https://mlsanity.com/paper/2503.08688/

## Key facts

- Cited by 54 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 11 Mar 2025.

## Built on by (most cited first, 30 of 54)

- [The Personality Illusion: Revealing Dissociation Between Self-Reports & Behavior in LLMs](https://mlsanity.com/paper/2509.03730/) (2025)
- [Cultivating Pluralism In Algorithmic Monoculture: The Community Alignment Dataset](https://mlsanity.com/paper/2507.09650/) (2025)
- [An Economy of AI Agents](https://mlsanity.com/paper/2509.01063/) (2025)
- [Pairwise Calibrated Rewards for Pluralistic Alignment](https://mlsanity.com/paper/2506.06298/) (2025)
- [Generative Value Conflicts Reveal LLM Priorities](https://mlsanity.com/paper/2509.25369/) (2025)
- [Fluent but Foreign: Even Regional LLMs Lack Cultural Alignment](https://mlsanity.com/paper/2505.21548/) (2025)
- [Meaning Is Not A Metric: Using LLMs to make cultural context legible at scale](https://mlsanity.com/paper/2505.23785/) (2025)
- [Alignment Revisited: Are Large Language Models Consistent in Stated and Revealed Preferences?](https://mlsanity.com/paper/2506.00751/) (2025)
- [The Singapore Consensus on Global AI Safety Research Priorities](https://mlsanity.com/paper/2506.20702/) (2025)
- [Culture is Everywhere: A Call for Intentionally Cultural Evaluation](https://mlsanity.com/paper/2509.01301/) (2025)
- [Silenced Biases: The Dark Side LLMs Learned to Refuse](https://mlsanity.com/paper/2511.03369/) (2025)
- [Real-World Gaps in AI Governance Research](https://mlsanity.com/paper/2505.00174/) (2025)
- [Building Benchmarks from the Ground Up: Community-Centered Evaluation of LLMs in Healthcare Chatbot Settings](https://mlsanity.com/paper/2509.24506/) (2025)
- [Zhyper: Factorized Hypernetworks for Conditioned LLM Fine-Tuning](https://mlsanity.com/paper/2510.19733/) (2025)
- [SpiritRAG: A Q&A System for Religion and Spirituality in the United Nations Archive](https://mlsanity.com/paper/2507.04395/) (2025)
- [Reward Models Inherit Value Biases from Pretraining](https://mlsanity.com/paper/2601.20838/) (2026)
- [Are Language Models Sensitive to Morally Irrelevant Distractors?](https://mlsanity.com/paper/2602.09416/) (2026)
- [Ambiguity Collapse by LLMs: A Taxonomy of Epistemic Risks](https://mlsanity.com/paper/2603.05801/) (2026)
- [Measuring Opinion Bias and Sycophancy via LLM-based Persuasion](https://mlsanity.com/paper/2604.21564/) (2026)
- [Probing Persona-Dependent Preferences in Language Models](https://mlsanity.com/paper/2605.13339/) (2026)
- [Reproducibility Study of "Cooperate or Collapse: Emergence of Sustainable Cooperation in a Society of LLM Agents"](https://mlsanity.com/paper/2505.09289/) (2025)
- [Improving the Distributional Alignment of LLMs using Supervision](https://mlsanity.com/paper/2507.00439/) (2025)
- [CRaFT: An Explanation-Based Framework for Evaluating Cultural Reasoning in Multilingual Language Models](https://mlsanity.com/paper/2510.14014/) (2025)
- [Reflect: Transparent Principle-Guided Reasoning for Constitutional Alignment at Scale](https://mlsanity.com/paper/2601.18730/) (2026)
- [Cultural Perspectives and Expectations for Generative AI: A Global Survey Approach](https://mlsanity.com/paper/2603.05723/) (2026)
- [Lessons Without Borders? Evaluating Cultural Alignment of LLMs Using Multilingual Story Moral Generation](https://mlsanity.com/paper/2604.08797/) (2026)
- [Amplify Initiative: Building A Localized Data Platform for Globalized AI](https://mlsanity.com/paper/2504.14105/) (2025)
- [MillStone: How Open-Minded Are LLMs?](https://mlsanity.com/paper/2509.11967/) (2025)
- [From Delegates to Trustees: How Optimizing for Long-Term Interests Shapes Bias and Alignment in LLM](https://mlsanity.com/paper/2510.12689/) (2025)
- [BengaliMoralBench: A Benchmark for Auditing Moral Reasoning in Large Language Models within Bengali Language and Culture](https://mlsanity.com/paper/2511.03180/) (2025)

## Abstract (arXiv)

Research on the 'cultural alignment' of Large Language Models (LLMs) has emerged in response to growing interest in understanding representation across diverse stakeholders. Current approaches to evaluating cultural alignment through survey-based assessments that borrow from social science methodologies often overlook systematic robustness checks. Here, we identify and test three assumptions behind current survey-based evaluation methods: (1) Stability: that cultural alignment is a property of LLMs rather than an artifact of evaluation design, (2) Extrapolability: that alignment with one culture on a narrow set of issues predicts alignment with that culture on others, and (3) Steerability: that LLMs can be reliably prompted to represent specific cultural perspectives. Through experiments examining both explicit and implicit preferences of leading LLMs, we find a high level of instability across presentation formats, incoherence between evaluated versus held-out cultural dimensions, and erratic behavior under prompt steering. We show that these inconsistencies can cause the results of an evaluation to be very sensitive to minor variations in methodology. Finally, we demonstrate in a case study on evaluation design that narrow experiments and a selective assessment of evidence can be used to paint an incomplete picture of LLMs' cultural alignment properties. Overall, these results highlight significant limitations of current survey-based approaches to evaluating the cultural alignment of LLMs and highlight a need for systematic robustness checks and red-teaming for evaluation results. Data and code are available at https://huggingface.co/datasets/akhan02/cultural-dimension-cover-letters and https://github.com/ariba-k/llm-cultural-alignment-evaluation, respectively.