mlsanityData to 7 Oct 2026

Knowledge of cultural moral norms in large language models

arXiv 2306.01857 ↗First posted 2 Jun 2023cs.CLcited by 51

Not on the map. Why?

Abstract (arXiv)

Moral norms vary across cultures. A recent line of work suggests that English large language models contain human-like moral biases, but these studies typically do not examine moral variation in a diverse cultural setting. We investigate the extent to which monolingual English language models contain knowledge about moral norms in different countries. We consider two levels of analysis: 1) whether language models capture fine-grained moral variation across countries over a variety of topics such as ``homosexuality'' and ``divorce''; 2) whether language models capture cultural diversity and shared tendencies in which topics people around the globe tend to diverge or agree on in their moral judgment. We perform our analyses with two public datasets from the World Values Survey (across 55 countries) and PEW global surveys (across 40 countries) on morality. We find that pre-trained English language models predict empirical moral norms across countries worse than the English moral norms reported previously. However, fine-tuning language models on the survey data improves inference across countries at the expense of a less accurate estimate of the English moral norms. We discuss the relevance and challenges of incorporating cultural knowledge into the automated inference of moral norms.

Key facts. Cited by 51 papers in the map (data through 7 Oct 2026). Builds on 24 of its 45 references that are in the map. First posted on arXiv 2 Jun 2023.

Builds on

24 of 45 references.

References · 24

  1. 2017Attention Is All You Needcited by 59,932
  2. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,204
  3. 2020Language Models are Few-Shot Learnerscited by 32,199
  4. 2019RoBERTa: A Robustly Optimized BERT Pretraining Approachcited by 13,266
  5. 2019Sentence-BERT: Sentence Embeddings using Siamese BERT-Networkscited by 9,049
  6. 2020Scaling Laws for Neural Language Modelscited by 6,411
  7. 2019Language Models as Knowledge Bases?cited by 1,922
  8. 2015Aligning Books and Movies: Towards Story-like Visual Explanations by Watching Movies and Reading Bookscited by 1,041
  9. 2020StereoSet: Measuring stereotypical bias in pretrained language modelscited by 911
  10. 2020Aligning AI With Shared Human Valuescited by 706
  11. 2019Social Bias Frames: Reasoning about Social and Power Implications of Languagecited by 377
  12. 2021Persistent Anti-Muslim Bias in Large Language Modelscited by 376
12 more
  1. 2020Social Chemistry 101: Learning to Reason about Social and Moral Normscited by 253
  2. 2021Large Pre-trained Language Models Contain Human-like Biases of What is Right and Wrong to Docited by 178
  3. 2022Probing Pre-Trained Language Models for Cross-Cultural Differences in Valuescited by 147
  4. 2020Moral Stories: Situated Reasoning about Norms, Intents, Actions, and their Consequencescited by 133
  5. 2020Scruples: A Corpus of Community Ethical Judgments on 32,000 Real-Life Anecdotescited by 100
  6. 2022Aligning to Social Norms and Values in Interactive Narrativescited by 32
  7. 2022GeoMLAMA: Geo-Diverse Commonsense Probing on Multilingual Pre-Trained Language Modelscited by 30
  8. 2021A Word on Machine Ethics: A Response to Jiang et al. (2021)cited by 23
  9. 2022The Moral Foundations Reddit Corpuscited by 20
  10. 2022Do Multilingual Language Models Capture Differing Moral Norms?cited by 15
  11. 2020Text-based inference of moral sentiment changecited by 11
  12. 2020When do Word Embeddings Accurately Reflect Surveys on our Beliefs About People?cited by 11

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2023: 6; 2024: 23; 2025: 13; 2026: 9.

51 papers.

Sort:

By area:

  1. 2024A Roadmap to Pluralistic Alignmentcited by 175
  2. 2023Cultural Bias and Cultural Alignment of Large Language Modelscited by 149
  3. 2024Emergent social conventions and collective bias in LLM populationscited by 115
  4. 2024CultureBank: An Online Community-Driven Knowledge Base Towards Culturally Aware Language Technologiescited by 98
  5. 2023A Survey on Fairness in Large Language Modelscited by 91
  6. 2024Spatial-Temporal Large Language Model for Traffic Predictioncited by 81
  7. 2024INCLUDE: Evaluating Multilingual Language Understanding with Regional Knowledgecited by 78
  8. 2023AceGPT, Localizing Large Language Models in Arabiccited by 75
  9. 2024Safeguarding Large Language Models: A Surveycited by 73
  10. 2024The Multilingual Alignment Prism: Aligning Global and Local Preferences to Reduce Harmcited by 67
  11. 2024Risk Taxonomy, Mitigation, and Assessment Benchmarks of Large Language Model Systemscited by 63
  12. 2024Building Guardrails for Large Language Modelscited by 61
38 more
  1. 2024CULTURE-GEN: Revealing Global Cultural Perception in Language Models through Natural Language Promptingcited by 55
  2. 2024Emergence of Social Norms in Generative Agent Societies: Principles and Architecturecited by 41
  3. 2023Rethinking Machine Ethics -- Can LLMs Perform Moral Reasoning through the Lens of Moral Theories?cited by 36
  4. 2024CulturalTeaming: AI-Assisted Interactive Red-Teaming for Challenging LLMs' (Lack of) Multicultural Knowledgecited by 28
  5. 2024LLM-GLOBE: A Benchmark Evaluating the Cultural Values Embedded in LLM Outputcited by 27
  6. 2024Culturally Aware and Adapted NLP: A Taxonomy and a Survey of the State of the Artcited by 26
  7. 2025Normative Evaluation of Large Language Models with Everyday Moral Dilemmascited by 23
  8. 2024High-Dimension Human Value Representation in Large Language Modelscited by 20
  9. 2023BHASA: A Holistic Southeast Asian Linguistic and Cultural Evaluation Suite for Large Language Modelscited by 16
  10. 2023Navigating Cultural Chasms: Exploring and Unlocking the Cultural POV of Text-To-Image Modelscited by 16
  11. 2024Position: Towards Bidirectional Human-AI Alignmentcited by 15
  12. 2025Beyond Words: Exploring Cultural Value Sensitivity in Multimodal Modelscited by 14
  13. 2024Creating a Lens of Chinese Culture: A Multimodal Dataset for Chinese Pun Rebus Art Understandingcited by 11
  14. 2025Lost in Cultural Translation: Do LLMs Struggle with Math Across Cultural Contexts?cited by 9
  15. 2024How Well Do LLMs Identify Cultural Unity in Diversity?cited by 8
  16. 2024Attributing Culture-Conditioned Generations to Pretraining Corporacited by 8
  17. 2025SaudiCulture: A Benchmark for Evaluating Large Language Models Cultural Competence within Saudi Arabiacited by 8
  18. 2024Unveiling and Mitigating Bias in Large Language Model Recommendations: A Path to Fairnesscited by 6
  19. 2026Tears or Cheers? Benchmarking LLMs via Culturally Elicited Distinct Affective Responsescited by 6
  20. 2024Can Agents Spontaneously Form a Society? Introducing a Novel Architecture for Generative Multi-Agents to Elicit Social Emergencecited by 5
  21. 2025Can (A)I Change Your Mind?cited by 5
  22. 2025Do Large Language Models Understand Morality Across Cultures?cited by 4
  23. 2025Designing Speech Technologies for Australian Aboriginal English: Opportunities, Risks and Participationcited by 3
  24. 2025Exploring Cultural Variations in Moral Judgments with Large Language Modelscited by 3
  25. 2026Culturally Grounded Personas in Large Language Models: Characterization and Alignment with Socio-Psychological Value Frameworkscited by 3
  26. 2024Runtime Monitoring and Enforcement of Conditional Fairness in Generative AIscited by 2
  27. 2025Does a Large Language Model Really Speak in Human-Like Language?cited by 1
  28. 2025Toward Socially Aware Vision-Language Models: Evaluating Cultural Competence Through Multimodal Story Generationcited by 1
  29. 2025Prompt Optimization Across Multiple Agents for Representing Diverse Human Populationscited by 1
  30. 2026When AI Takes Sides on Questions of Faith: Persistent Asymmetries in AI-Mediated Faith Guidancecited by 1
  31. 2024GPT-4's One-Dimensional Mapping of Morality: How the Accuracy of Country-Estimates Depends on Moral Domaincited by 0
  32. 2025Visual moral inference and communicationcited by 0
  33. 2025Dissecting Physics Reasoning in Small Language Models: A Multi-Dimensional Analysis from an Educational Perspectivecited by 0
  34. 2026Does Claude's Constitution Have a Culture?cited by 0
  35. 2026How Value Induction Reshapes LLM Behaviourcited by 0
  36. 2026PAVE: A Cognitive Architecture for Legitimate Violation in Generative Agent Societiescited by 0
  37. 2026CCBENCH: Assessing LLM Cultural Competence via Implicitly Signaled Norms using Health Queriescited by 0
  38. 2026MemeBridge: A Dataset for Benchmarking and Mitigating the Bidirectional Cultural Gap in Meme Interpretationcited by 0

50 most cited of 51 All 51 (JSON)