# An Evaluation of Cultural Value Alignment in LLM

arXiv: https://arxiv.org/abs/2504.08863 · HTML: https://mlsanity.com/paper/2504.08863/

## Key facts

- Cited by 28 papers in the map (data through 14 Oct 2026).
- Builds on 23 of its 33 references that are in the map.
- First posted on arXiv 11 Apr 2025.

## Builds on (most cited first, 23 of 23)

- [LLaMA: Open and Efficient Foundation Language Models](https://mlsanity.com/paper/2302.13971/) (2023)
- [GPT-4o System Card](https://mlsanity.com/paper/2410.21276/) (2024)
- [DeepSeek-V2: A Strong, Economical, and Efficient Mixture-of-Experts Language Model](https://mlsanity.com/paper/2405.04434/) (2024)
- [GLM: General Language Model Pretraining with Autoregressive Blank Infilling](https://mlsanity.com/paper/2103.10360/) (2021)
- [Language (Technology) is Power: A Critical Survey of "Bias" in NLP](https://mlsanity.com/paper/2005.14050/) (2020)
- [Whose Opinions Do Language Models Reflect?](https://mlsanity.com/paper/2303.17548/) (2023)
- [Towards Measuring the Representation of Subjective Global Opinions in Language Models](https://mlsanity.com/paper/2306.16388/) (2023)
- [BOLD: Dataset and Metrics for Measuring Biases in Open-Ended Language Generation](https://mlsanity.com/paper/2101.11718/) (2021)
- [Persistent Anti-Muslim Bias in Large Language Models](https://mlsanity.com/paper/2101.05783/) (2021)
- [Towards Understanding and Mitigating Social Biases in Language Models](https://mlsanity.com/paper/2106.13219/) (2021)
- [The Alignment Problem from a Deep Learning Perspective](https://mlsanity.com/paper/2209.00626/) (2022)
- [Assessing Cross-Cultural Alignment between ChatGPT and Human Societies: An Empirical Study](https://mlsanity.com/paper/2303.17466/) (2023)
- [BLEnD: A Benchmark for LLMs on Everyday Knowledge in Diverse Cultures and Languages](https://mlsanity.com/paper/2406.09948/) (2024)
- [Having Beer after Prayer? Measuring Cultural Bias in Large Language Models](https://mlsanity.com/paper/2305.14456/) (2023)
- [Probing Pre-Trained Language Models for Cross-Cultural Differences in Values](https://mlsanity.com/paper/2203.13722/) (2022)
- [Co-Writing with Opinionated Language Models Affects Users' Views](https://mlsanity.com/paper/2302.00560/) (2023)
- [Investigating Cultural Alignment of Large Language Models](https://mlsanity.com/paper/2402.13231/) (2024)
- [Cultural Alignment in Large Language Models: An Explanatory Analysis Based on Hofstede's Cultural Dimensions](https://mlsanity.com/paper/2309.12342/) (2023)
- [Generative Language Models Exhibit Social Identity Biases](https://mlsanity.com/paper/2310.15819/) (2023)
- [The Ghost in the Machine has an American accent: value conflict in GPT-3](https://mlsanity.com/paper/2203.07785/) (2022)
- [Unintended Impacts of LLM Alignment on Global Representation](https://mlsanity.com/paper/2402.15018/) (2024)
- [Large Language Models Reflect the Ideology of their Creators](https://mlsanity.com/paper/2410.18417/) (2024)
- [SafeWorld: Geo-Diverse Safety Alignment](https://mlsanity.com/paper/2412.06483/) (2024)

## Built on by (most cited first, 28 of 28)

- [On the Alignment of Large Language Models with Global Human Opinion](https://mlsanity.com/paper/2509.01418/) (2025)
- [CURE: Cultural Understanding and Reasoning Evaluation - A Framework for "Thick" Culture Alignment Evaluation in LLMs](https://mlsanity.com/paper/2511.12014/) (2025)
- [Evaluating LLM Adaptation to Sociodemographic Factors: User Profile vs. Dialogue History](https://mlsanity.com/paper/2505.21362/) (2025)
- [Building Benchmarks from the Ground Up: Community-Centered Evaluation of LLMs in Healthcare Chatbot Settings](https://mlsanity.com/paper/2509.24506/) (2025)
- [The Behavioral Fabric of LLM-Powered GUI Agents: Human Values and Interaction Outcomes](https://mlsanity.com/paper/2601.16356/) (2026)
- [Seeing Symbols, Missing Cultures: Probing Vision-Language Models' Reasoning on Fire Imagery and Cultural Meaning](https://mlsanity.com/paper/2509.23311/) (2025)
- [On the Credibility of Evaluating LLMs using Survey Questions](https://mlsanity.com/paper/2602.04033/) (2026)
- [Mind the Gap in Cultural Alignment: Task-Aware Culture Management for Large Language Models](https://mlsanity.com/paper/2602.22475/) (2026)
- [CRaFT: An Explanation-Based Framework for Evaluating Cultural Reasoning in Multilingual Language Models](https://mlsanity.com/paper/2510.14014/) (2025)
- [A Unified Framework to Quantify Cultural Intelligence of AI](https://mlsanity.com/paper/2603.01211/) (2026)
- [Implicit Values Embedded in How Humans and LLMs Complete Subjective Everyday Tasks](https://mlsanity.com/paper/2510.03384/) (2025)
- [Neuron-Level Analysis of Cultural Understanding in Large Language Models](https://mlsanity.com/paper/2510.08284/) (2025)
- [Prioritize Economy or Climate Action? Investigating ChatGPT Response Differences Based on Inferred Political Orientation](https://mlsanity.com/paper/2511.04706/) (2025)
- [Mitigating Cultural Bias in LLMs via Multi-Agent Cultural Debate](https://mlsanity.com/paper/2601.12091/) (2026)
- [VISPA: Pluralistic Alignment via Automatic Value Selection and Activation](https://mlsanity.com/paper/2601.12758/) (2026)
- [Language Models Entangle Language and Culture](https://mlsanity.com/paper/2601.15337/) (2026)
- [Distributional Open-Ended Evaluation of LLM Cultural Value Alignment Based on Value Codebook](https://mlsanity.com/paper/2604.06210/) (2026)
- [People Are Not Just Their Countries. Disentangling Social Determinants of LLM Value Alignment Across Europe](https://mlsanity.com/paper/2608.07367/) (2026)
- ['Too much alignment; not enough culture': Re-balancing cultural alignment practices in LLMs](https://mlsanity.com/paper/2509.26167/) (2025)
- [CuMA: Aligning LLMs with Sparse Cultural Values via Demographic-Aware Mixture of Adapters](https://mlsanity.com/paper/2601.04885/) (2026)
- [Beyond Instrumental and Substitutive Paradigms: Introducing Machine Culture as an Emergent Phenomenon in Large Language Models](https://mlsanity.com/paper/2601.17096/) (2026)
- [Are Aligned Large Language Models Still Misaligned?](https://mlsanity.com/paper/2602.11305/) (2026)
- [AdaCultureSafe: Adaptive Cultural Safety Grounded by Cultural Knowledge in Large Language Models](https://mlsanity.com/paper/2603.08275/) (2026)
- [Modeling Community Attitude through Reaction Tone: A Human-AI Collaborative Framework for Evaluating LLM Alignment with Linguistic Behaviors in Online Communities](https://mlsanity.com/paper/2605.27388/) (2026)
- [ExCAM: Explainable Cultural Awareness Metrics](https://mlsanity.com/paper/2605.29897/) (2026)
- [Toward Responsible and Epistemically Grounded Multilingual LLMs for Computational Social Science and Humanities](https://mlsanity.com/paper/2606.00596/) (2026)
- [The Shibboleth Effect: Auditing the Cross-Lingual Distributional Skew of Large Language Models](https://mlsanity.com/paper/2606.11082/) (2026)
- [Alignment by Stereotyping: How LLMs Sacrifice Individual Distinctiveness for Cultural Adaptation](https://mlsanity.com/paper/2609.05993/) (2026)

## Abstract (arXiv)

LLMs as intelligent agents are being increasingly applied in scenarios where human interactions are involved, leading to a critical concern about whether LLMs are faithful to the variations in culture across regions. Several works have investigated this question in various ways, finding that there are biases present in the cultural representations of LLM outputs. To gain a more comprehensive view, in this work, we conduct the first large-scale evaluation of LLM culture assessing 20 countries' cultures and languages across ten LLMs. With a renowned cultural values questionnaire and by carefully analyzing LLM output with human ground truth scores, we thoroughly study LLMs' cultural alignment across countries and among individual models. Our findings show that the output over all models represents a moderate cultural middle ground. Given the overall skew, we propose an alignment metric, revealing that the United States is the best-aligned country and GLM-4 has the best ability to align to cultural values. Deeper investigation sheds light on the influence of model origin, prompt language, and value dimensions on cultural output. Specifically, models, regardless of where they originate, align better with the US than they do with China. The conclusions provide insight to how LLMs can be better aligned to various cultures as well as provoke further discussion of the potential for LLMs to propagate cultural bias and the need for more culturally adaptable models.