mlsanityData to 7 Oct 2026

Investigating Cultural Alignment of Large Language Models

arXiv 2402.13231 ↗First posted 20 Feb 2024cs.CL cs.CYcited by 140

On the map: LLM alignment / RLHF / preference optimisation → Human values and AI feedback alignment

Abstract (arXiv)

The intricate relationship between language and culture has long been a subject of exploration within the realm of linguistic anthropology. Large Language Models (LLMs), promoted as repositories of collective human knowledge, raise a pivotal question: do these models genuinely encapsulate the diverse knowledge adopted by different cultures? Our study reveals that these models demonstrate greater cultural alignment along two dimensions -- firstly, when prompted with the dominant language of a specific culture, and secondly, when pretrained with a refined mixture of languages employed by that culture. We quantify cultural alignment by simulating sociological surveys, comparing model responses to those of actual survey participants as references. Specifically, we replicate a survey conducted in various regions of Egypt and the United States through prompting LLMs with different pretraining data mixtures in both Arabic and English with the personas of the real respondents and the survey questions. Further analysis reveals that misalignment becomes more pronounced for underrepresented personas and for culturally sensitive topics, such as those probing social values. Finally, we introduce Anthropological Prompting, a novel method leveraging anthropological reasoning to enhance cultural alignment. Our study emphasizes the necessity for a more balanced multilingual pretraining dataset to better represent the diversity of human experience and the plurality of different cultures with many implications on the topic of cross-lingual transfer.

Key facts. Cited by 140 papers in the map (data through 7 Oct 2026). Placed in Human values and AI feedback alignment, one of 10 directions in LLM alignment / RLHF / preference optimisation. Builds on 19 of its 29 references that are in the map. First posted on arXiv 20 Feb 2024.

Builds on

19 of 29 references.

References · 19

  1. 2019The Woman Worked as a Babysitter: On Biases in Language Generationcited by 540
  2. 2023Towards Measuring the Representation of Subjective Global Opinions in Language Modelscited by 424
  3. 2021Persistent Anti-Muslim Bias in Large Language Modelscited by 376
  4. 2023Toxicity in ChatGPT: Analyzing Persona-assigned Language Modelscited by 356
  5. 2021Few-shot Learning with Multilingual Language Modelscited by 278
  6. 2023Assessing Cross-Cultural Alignment between ChatGPT and Human Societies: An Empirical Studycited by 203
  7. 2021Societal Biases in Language Generation: Progress and Challengescited by 201
  8. 2020Social Biases in NLP Models as Barriers for Persons with Disabilitiescited by 182
  9. 2023Do LLMs exhibit human-like response biases? A case study in survey designcited by 176
  10. 2022Probing Pre-Trained Language Models for Cross-Cultural Differences in Valuescited by 147
  11. 2022Lifting the Curse of Multilinguality by Pre-training Modular Transformerscited by 145
  12. 2023Cross-Lingual Consistency of Factual Knowledge in Multilingual Language Modelscited by 117
7 more
  1. 2023Nationality Bias in Text Generationcited by 50
  2. 2023Improving Diversity of Demographic Representation in Large Language Models via Collective-Critiques and Self-Votingcited by 46
  3. 2023Personas as a Way to Model Truthfulness in Language Modelscited by 46
  4. 2023GPTAraEval: A Comprehensive Evaluation of ChatGPT on Arabic NLPcited by 37
  5. 2023This Land is {Your, My} Land: Evaluating Geopolitical Biases in Language Modelscited by 31
  6. 2022Discovering Language-neutral Sub-networks in Multilingual Language Modelscited by 28
  7. 2023Taqyim: Evaluating Arabic NLP Tasks Using ChatGPT Modelscited by 10

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2024: 24; 2025: 58; 2026: 58.

140 papers.

Sort:

By area:

  1. 2024The PRISM Alignment Dataset: What Participatory, Representative and Individualised Human Feedback Reveals About the Subjective and Multicultural Alignment of Large Language Modelscited by 136
  2. 2024Aligning to Thousands of Preferences via System Message Generalizationcited by 84
  3. 2024INCLUDE: Evaluating Multilingual Language Understanding with Regional Knowledgecited by 78
  4. 2024CULTURE-GEN: Revealing Global Cultural Perception in Language Models through Natural Language Promptingcited by 55
  5. 2025Specializing Large Language Models to Simulate Survey Response Distributions for Global Populationscited by 50
  6. 2024AraDiCE: Benchmarks for Dialectal and Cultural Capabilities in LLMscited by 46
  7. 2025Culture is Not Trivia: Sociocultural Theory for Cultural NLPcited by 40
  8. 2025Multilingual != Multicultural: Evaluating Gaps Between Multilingual Capabilities and Cultural Alignment in LLMscited by 39
  9. 2024Controllable Safety Alignment: Inference-Time Adaptation to Diverse Safety Requirementscited by 35
  10. 2024CaLMQA: Exploring culturally specific long-form question answering across 23 languagescited by 31
  11. 2024NativQA: Multilingual Culturally-Aligned Natural Query for LLMscited by 30
  12. 2025An Evaluation of Cultural Value Alignment in LLMcited by 28
38 more
  1. 2024LLM-GLOBE: A Benchmark Evaluating the Cultural Values Embedded in LLM Outputcited by 27
  2. 2024Culturally Aware and Adapted NLP: A Taxonomy and a Survey of the State of the Artcited by 26
  3. 2024SysBench: Can Large Language Models Follow System Messages?cited by 26
  4. 2024Understanding Long Videos with Multimodal Language Modelscited by 25
  5. 2024Extrinsic Evaluation of Cultural Competence in Large Language Modelscited by 24
  6. 2024BertaQA: How Much Do Language Models Know About Local Culture?cited by 23
  7. 2024Preference Tuning with Human Feedback on Language, Speech, and Vision Tasks: A Surveycited by 23
  8. 2024High-Dimension Human Value Representation in Large Language Modelscited by 20
  9. 2025Multiple LLM Agents Debate for Equitable Cultural Alignmentcited by 20
  10. 2025Localized Cultural Knowledge is Conserved and Controllable in Large Language Modelscited by 19
  11. 2025Synthetic Socratic Debates: Examining Persona Effects on Moral Decision and Persuasion Dynamicscited by 19
  12. 2024Multilingual Large Language Models: A Systematic Surveycited by 18
  13. 2024See It from My Perspective: How Language Affects Cultural Bias in Image Understandingcited by 17
  14. 2025Cultural Learning-Based Culture Adaptation of Language Modelscited by 16
  15. 2025Grounding Multilingual Multimodal LLMs With Cultural Knowledgecited by 16
  16. 2024Position: Towards Bidirectional Human-AI Alignmentcited by 15
  17. 2025Palm: A Culturally Inclusive and Linguistically Diverse Dataset for Arabic LLMscited by 15
  18. 2025CulFiT: A Fine-grained Cultural-aware LLM Training Paradigm via Multilingual Critique Data Synthesiscited by 14
  19. 2025MIRROR: Multimodal Cognitive Reframing Therapy for Rolling with Resistancecited by 13
  20. 2025NileChat: Towards Linguistically Diverse and Culturally Aware LLMs for Local Communitiescited by 13
  21. 2025Hire Your Anthropologist! Rethinking Culture Benchmarks Through an Anthropological Lenscited by 13
  22. 2025ValuesRAG: Enhancing Cultural Alignment Through Retrieval-Augmented Contextual Learningcited by 10
  23. 2025Time Travel: A Comprehensive Benchmark to Evaluate LMMs on Historical and Cultural Artifactscited by 10
  24. 2025TALES: A Taxonomy and Analysis of Cultural Representations in LLM-generated Storiescited by 10
  25. 2024Cultural Adaptation of Menus: A Fine-Grained Approachcited by 8
  26. 2025PerCul: A Story-Driven Cultural Evaluation of LLMs in Persiancited by 8
  27. 2025Provocations from the Humanities for Generative AI Researchcited by 8
  28. 2025SaudiCulture: A Benchmark for Evaluating Large Language Models Cultural Competence within Saudi Arabiacited by 8
  29. 2025Revisiting LLM Value Probing Strategies: Are They Robust and Expressive?cited by 8
  30. 2026Mechanistic Interpretability for Large Language Model Alignment: Progress, Challenges, and Future Directionscited by 8
  31. 2025Analyzing Political Bias in LLMs via Target-Oriented Sentiment Classificationcited by 7
  32. 2025Benchmarking Overton Pluralism in LLMscited by 7
  33. 2026Assessing the Reliability of Persona-Conditioned LLMs as Synthetic Survey Respondentscited by 7
  34. 2025Value Compass Benchmarks: A Platform for Fundamental and Validated Evaluation of LLMs Valuescited by 6
  35. 2025LLM Alignment for the Arabs: A Homogenous Culture or Diverse Ones?cited by 6
  36. 2025We Politely Insist: Your LLM Must Learn the Persian Art of Taarofcited by 6
  37. 2025CURE: Cultural Understanding and Reasoning Evaluation - A Framework for "Thick" Culture Alignment Evaluation in LLMscited by 6
  38. 2026The Chameleon's Limit: Investigating Persona Collapse and Homogenization in Large Language Modelscited by 6

50 most cited of 140 All 140 (JSON)

Similar, not cited

Close in wording, no citation either way

  1. 2026AlignCultura: Towards Culturally Aligned Large Language Models?
  2. 2023Cultural Alignment in Large Language Models: An Explanatory Analysis Based on Hofstede's Cultural Dimensions
  3. 2026Aligned but Flattened: Analyzing the Trade-off between Cultural Alignment and Diversity in LLMs
  4. 2024Survey of Cultural Awareness in Language Models: Text and Beyond
  5. 2023CDEval: A Benchmark for Measuring the Cultural Dimensions of Large Language Models
  6. 2025Break the Checkbox: Challenging Closed-Style Evaluations of Cultural Alignment in LLMs
  7. 2025The American Ghost in the Machine: How language models align culturally and the effects of cultural prompting
  8. 2025Randomness, Not Representation: The Unreliability of Evaluating Cultural Alignment in LLMs