# Investigating Cultural Alignment of Large Language Models

arXiv: https://arxiv.org/abs/2402.13231 · HTML: https://mlsanity.com/paper/2402.13231/

## Key facts

- Cited by 143 papers in the map (data through 14 Oct 2026).
- Placed in Human values and AI feedback alignment, one of 10 directions in LLM alignment / RLHF / preference optimisation.
- Builds on 19 of its 29 references that are in the map.
- First posted on arXiv 20 Feb 2024.

## Placed in

- LLM alignment / RLHF / preference optimisation / [Human values and AI feedback alignment](https://mlsanity.com/alignment/human-values-and-ai-feedback-alignment/)

## Builds on (most cited first, 19 of 19)

- [The Woman Worked as a Babysitter: On Biases in Language Generation](https://mlsanity.com/paper/1909.01326/) (2019)
- [Towards Measuring the Representation of Subjective Global Opinions in Language Models](https://mlsanity.com/paper/2306.16388/) (2023)
- [Persistent Anti-Muslim Bias in Large Language Models](https://mlsanity.com/paper/2101.05783/) (2021)
- [Toxicity in ChatGPT: Analyzing Persona-assigned Language Models](https://mlsanity.com/paper/2304.05335/) (2023)
- [Few-shot Learning with Multilingual Language Models](https://mlsanity.com/paper/2112.10668/) (2021)
- [Assessing Cross-Cultural Alignment between ChatGPT and Human Societies: An Empirical Study](https://mlsanity.com/paper/2303.17466/) (2023)
- [Societal Biases in Language Generation: Progress and Challenges](https://mlsanity.com/paper/2105.04054/) (2021)
- [Social Biases in NLP Models as Barriers for Persons with Disabilities](https://mlsanity.com/paper/2005.00813/) (2020)
- [Do LLMs exhibit human-like response biases? A case study in survey design](https://mlsanity.com/paper/2311.04076/) (2023)
- [Probing Pre-Trained Language Models for Cross-Cultural Differences in Values](https://mlsanity.com/paper/2203.13722/) (2022)
- [Lifting the Curse of Multilinguality by Pre-training Modular Transformers](https://mlsanity.com/paper/2205.06266/) (2022)
- [Cross-Lingual Consistency of Factual Knowledge in Multilingual Language Models](https://mlsanity.com/paper/2310.10378/) (2023)
- [Nationality Bias in Text Generation](https://mlsanity.com/paper/2302.02463/) (2023)
- [Personas as a Way to Model Truthfulness in Language Models](https://mlsanity.com/paper/2310.18168/) (2023)
- [Improving Diversity of Demographic Representation in Large Language Models via Collective-Critiques and Self-Voting](https://mlsanity.com/paper/2310.16523/) (2023)
- [GPTAraEval: A Comprehensive Evaluation of ChatGPT on Arabic NLP](https://mlsanity.com/paper/2305.14976/) (2023)
- [This Land is {Your, My} Land: Evaluating Geopolitical Biases in Language Models](https://mlsanity.com/paper/2305.14610/) (2023)
- [Discovering Language-neutral Sub-networks in Multilingual Language Models](https://mlsanity.com/paper/2205.12672/) (2022)
- [Taqyim: Evaluating Arabic NLP Tasks Using ChatGPT Models](https://mlsanity.com/paper/2306.16322/) (2023)

## Built on by (most cited first, 30 of 143)

- [The PRISM Alignment Dataset: What Participatory, Representative and Individualised Human Feedback Reveals About the Subjective and Multicultural Alignment of Large Language Models](https://mlsanity.com/paper/2404.16019/) (2024)
- [INCLUDE: Evaluating Multilingual Language Understanding with Regional Knowledge](https://mlsanity.com/paper/2411.19799/) (2024)
- [Aligning to Thousands of Preferences via System Message Generalization](https://mlsanity.com/paper/2405.17977/) (2024)
- [CULTURE-GEN: Revealing Global Cultural Perception in Language Models through Natural Language Prompting](https://mlsanity.com/paper/2404.10199/) (2024)
- [Specializing Large Language Models to Simulate Survey Response Distributions for Global Populations](https://mlsanity.com/paper/2502.07068/) (2025)
- [AraDiCE: Benchmarks for Dialectal and Cultural Capabilities in LLMs](https://mlsanity.com/paper/2409.11404/) (2024)
- [Culture is Not Trivia: Sociocultural Theory for Cultural NLP](https://mlsanity.com/paper/2502.12057/) (2025)
- [Multilingual != Multicultural: Evaluating Gaps Between Multilingual Capabilities and Cultural Alignment in LLMs](https://mlsanity.com/paper/2502.16534/) (2025)
- [Controllable Safety Alignment: Inference-Time Adaptation to Diverse Safety Requirements](https://mlsanity.com/paper/2410.08968/) (2024)
- [CaLMQA: Exploring culturally specific long-form question answering across 23 languages](https://mlsanity.com/paper/2406.17761/) (2024)
- [NativQA: Multilingual Culturally-Aligned Natural Query for LLMs](https://mlsanity.com/paper/2407.09823/) (2024)
- [SysBench: Can Large Language Models Follow System Messages?](https://mlsanity.com/paper/2408.10943/) (2024)
- [An Evaluation of Cultural Value Alignment in LLM](https://mlsanity.com/paper/2504.08863/) (2025)
- [LLM-GLOBE: A Benchmark Evaluating the Cultural Values Embedded in LLM Output](https://mlsanity.com/paper/2411.06032/) (2024)
- [Culturally Aware and Adapted NLP: A Taxonomy and a Survey of the State of the Art](https://mlsanity.com/paper/2406.03930/) (2024)
- [Understanding Long Videos with Multimodal Language Models](https://mlsanity.com/paper/2403.16998/) (2024)
- [Extrinsic Evaluation of Cultural Competence in Large Language Models](https://mlsanity.com/paper/2406.11565/) (2024)
- [BertaQA: How Much Do Language Models Know About Local Culture?](https://mlsanity.com/paper/2406.07302/) (2024)
- [Preference Tuning with Human Feedback on Language, Speech, and Vision Tasks: A Survey](https://mlsanity.com/paper/2409.11564/) (2024)
- [Multiple LLM Agents Debate for Equitable Cultural Alignment](https://mlsanity.com/paper/2505.24671/) (2025)
- [High-Dimension Human Value Representation in Large Language Models](https://mlsanity.com/paper/2404.07900/) (2024)
- [Localized Cultural Knowledge is Conserved and Controllable in Large Language Models](https://mlsanity.com/paper/2504.10191/) (2025)
- [Synthetic Socratic Debates: Examining Persona Effects on Moral Decision and Persuasion Dynamics](https://mlsanity.com/paper/2506.12657/) (2025)
- [Position: Towards Bidirectional Human-AI Alignment](https://mlsanity.com/paper/2406.09264/) (2024)
- [Multilingual Large Language Models: A Systematic Survey](https://mlsanity.com/paper/2411.11072/) (2024)
- [See It from My Perspective: How Language Affects Cultural Bias in Image Understanding](https://mlsanity.com/paper/2406.11665/) (2024)
- [Cultural Learning-Based Culture Adaptation of Language Models](https://mlsanity.com/paper/2504.02953/) (2025)
- [Grounding Multilingual Multimodal LLMs With Cultural Knowledge](https://mlsanity.com/paper/2508.07414/) (2025)
- [Palm: A Culturally Inclusive and Linguistically Diverse Dataset for Arabic LLMs](https://mlsanity.com/paper/2503.00151/) (2025)
- [CulFiT: A Fine-grained Cultural-aware LLM Training Paradigm via Multilingual Critique Data Synthesis](https://mlsanity.com/paper/2505.19484/) (2025)

## Abstract (arXiv)

The intricate relationship between language and culture has long been a subject of exploration within the realm of linguistic anthropology. Large Language Models (LLMs), promoted as repositories of collective human knowledge, raise a pivotal question: do these models genuinely encapsulate the diverse knowledge adopted by different cultures? Our study reveals that these models demonstrate greater cultural alignment along two dimensions -- firstly, when prompted with the dominant language of a specific culture, and secondly, when pretrained with a refined mixture of languages employed by that culture. We quantify cultural alignment by simulating sociological surveys, comparing model responses to those of actual survey participants as references. Specifically, we replicate a survey conducted in various regions of Egypt and the United States through prompting LLMs with different pretraining data mixtures in both Arabic and English with the personas of the real respondents and the survey questions. Further analysis reveals that misalignment becomes more pronounced for underrepresented personas and for culturally sensitive topics, such as those probing social values. Finally, we introduce Anthropological Prompting, a novel method leveraging anthropological reasoning to enhance cultural alignment. Our study emphasizes the necessity for a more balanced multilingual pretraining dataset to better represent the diversity of human experience and the plurality of different cultures with many implications on the topic of cross-lingual transfer.