mlsanityData to 14 Oct 2026

Assessing Cross-Cultural Alignment between ChatGPT and Human Societies: An Empirical Study

arXiv 2303.17466 ↗First posted 30 Mar 2023cs.CLcited by 206

Not on the map. Why?

Cited by 21 in LLM alignment / RLHF / preference optimisation, but placed by its own abstract, which didn’t fit.

Abstract (arXiv)

The recent release of ChatGPT has garnered widespread recognition for its exceptional ability to generate human-like responses in dialogue. Given its usage by users from various nations and its training on a vast multilingual corpus that incorporates diverse cultural and societal norms, it is crucial to evaluate its effectiveness in cultural adaptation. In this paper, we investigate the underlying cultural background of ChatGPT by analyzing its responses to questions designed to quantify human cultural differences. Our findings suggest that, when prompted with American context, ChatGPT exhibits a strong alignment with American culture, but it adapts less effectively to other cultural contexts. Furthermore, by using different prompts to probe the model, we show that English prompts reduce the variance in model responses, flattening out cultural differences and biasing them towards American culture. This study provides valuable insights into the cultural implications of ChatGPT and highlights the necessity of greater diversity and cultural awareness in language technologies.

Key facts. Cited by 206 papers in the map (data through 14 Oct 2026). Builds on 20 of its 45 references that are in the map. First posted on arXiv 30 Mar 2023.

Builds on

20 of 45 references.

References · 20

  1. 2020Language Models are Few-Shot Learnerscited by 32,419
  2. 2022Training language models to follow instructions with human feedbackcited by 15,971
  3. 2017Deep reinforcement learning from human preferencescited by 3,912
  4. 2023A Multitask, Multilingual, Multimodal Evaluation of ChatGPT on Reasoning, Hallucination, and Interactivitycited by 894
  5. 2023Is ChatGPT a General-Purpose Natural Language Processing Task Solver?cited by 488
  6. 2021Measuring and Improving Consistency in Pretrained Language Modelscited by 364
  7. 2022Challenges and Strategies in Cross-Cultural NLPcited by 218
  8. 2021Visually Grounded Reasoning across Languages and Culturescited by 189
  9. 2020Social Biases in NLP Models as Barriers for Persons with Disabilitiescited by 182
  10. 2021Large Pre-trained Language Models Contain Human-like Biases of What is Right and Wrong to Docited by 178
  11. 2022Probing Pre-Trained Language Models for Cross-Cultural Differences in Valuescited by 149
  12. 2022The Ghost in the Machine has an American accent: value conflict in GPT-3cited by 105
8 more
  1. 2021Language Models are Few-shot Multilingual Learnerscited by 92
  2. 2020Measuring Social Biases in Grounded Vision and Language Embeddingscited by 61
  3. 2021Stereotype and Skew: Quantifying Gender Bias in Pre-trained and Fine-tuned Language Modelscited by 57
  4. 2022Speaking Multiple Languages Affects the Moral Bias of Language Modelscited by 34
  5. 2022Does Moral Code Have a Moral Code? Probing Delphi's Moral Philosophycited by 31
  6. 2022Factual Consistency of Multilingual Pretrained Language Modelscited by 15
  7. 2022HERB: Measuring Hierarchical Regional Bias in Pre-trained Language Modelscited by 8
  8. 2022EnCBP: A New Benchmark Dataset for Finer-Grained Cultural Background Prediction in Englishcited by 7

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2023: 24; 2024: 63; 2025: 57; 2026: 62.

206 papers.

Sort:

By area:

  1. 2023A Survey on Evaluation of Large Language Modelscited by 1,242
  2. 2023Trustworthy LLMs: a Survey and Guideline for Evaluating Large Language Models' Alignmentcited by 401
  3. 2023Large Language Model as Attributed Training Data Generator: A Tale of Diversity and Biascited by 232
  4. 2024Leak, Cheat, Repeat: Data Contamination and Evaluation Malpractices in Closed-Source LLMscited by 229
  5. 2024Towards Measuring and Modeling "Culture" in LLMs: A Surveycited by 167
  6. 2024Monitoring AI-Modified Content at Scale: A Case Study on the Impact of ChatGPT on AI Conference Peer Reviewscited by 164
  7. 2023Having Beer after Prayer? Measuring Cultural Bias in Large Language Modelscited by 153
  8. 2023Cultural Bias and Cultural Alignment of Large Language Modelscited by 149
  9. 2024Investigating Cultural Alignment of Large Language Modelscited by 143
  10. 2023Cultural Alignment in Large Language Models: An Explanatory Analysis Based on Hofstede's Cultural Dimensionscited by 112
  11. 2024CultureBank: An Online Community-Driven Knowledge Base Towards Culturally Aware Language Technologiescited by 98
  12. 2024Risk Taxonomy, Mitigation, and Assessment Benchmarks of Large Language Model Systemscited by 66
38 more
  1. 2024AI Suggestions Homogenize Writing Toward Western Styles and Diminish Cultural Nuancescited by 60
  2. 2024DailyDilemmas: Revealing Value Preferences of LLMs with Quandaries of Daily Lifecited by 59
  3. 2024Are Large Language Models Consistent over Value-laden Questions?cited by 58
  4. 2023Multilingual Language Models are not Multicultural: A Case Study in Emotioncited by 54
  5. 2025Specializing Large Language Models to Simulate Survey Response Distributions for Global Populationscited by 53
  6. 2024CultureLLM: Incorporating Cultural Differences into Large Language Modelscited by 50
  7. 2023Large Language Models as Superpositions of Cultural Perspectivescited by 49
  8. 2025CultureVLM: Characterizing and Improving Cultural Understanding of Vision-Language Models for over 100 Countriescited by 42
  9. 2023LatEval: An Interactive LLMs Evaluation Benchmark with Incomplete Information from Lateral Thinking Puzzlescited by 41
  10. 2024Should We Respect LLMs? A Cross-Lingual Study on the Influence of Prompt Politeness on LLM Performancecited by 41
  11. 2023CHEAT: A Large-scale Dataset for Detecting ChatGPT-writtEn AbsTractscited by 40
  12. 2025Culture is Not Trivia: Sociocultural Theory for Cultural NLPcited by 40
  13. 2025Multilingual != Multicultural: Evaluating Gaps Between Multilingual Capabilities and Cultural Alignment in LLMscited by 39
  14. 2024Evaluating Cultural Adaptability of a Large Language Model via Simulation of Synthetic Personascited by 37
  15. 2024KorNAT: LLM Alignment Benchmark for Korean Social Values and Common Knowledgecited by 33
  16. 2023This Land is {Your, My} Land: Evaluating Geopolitical Biases in Language Modelscited by 32
  17. 2023Are Multilingual LLMs Culturally-Diverse Reasoners? An Investigation into Multicultural Proverbs and Sayingscited by 32
  18. 2024Self-Pluralising Culture Alignment for Large Language Modelscited by 31
  19. 2023A Fast Optimization View: Reformulating Single Layer Attention in LLM Based on Tensor and SVM Trick, and Solving It in Matrix Multiplication Timecited by 30
  20. 2024CulturalBench: A Robust, Diverse, and Challenging Cultural Benchmark by Human-AI CulturalTeamingcited by 30
  21. 2025Show Me the Work: Fact-Checkers' Requirements for Explainable Automated Fact-Checkingcited by 29
  22. 2024CulturalTeaming: AI-Assisted Interactive Red-Teaming for Challenging LLMs' (Lack of) Multicultural Knowledgecited by 28
  23. 2025An Evaluation of Cultural Value Alignment in LLMcited by 28
  24. 2023Denevil: Towards Deciphering and Navigating the Ethical Values of Large Language Models via Instruction Learningcited by 26
  25. 2024Culturally Aware and Adapted NLP: A Taxonomy and a Survey of the State of the Artcited by 26
  26. 2025Break the Checkbox: Challenging Closed-Style Evaluations of Cultural Alignment in LLMscited by 26
  27. 2024"They are uncultured": Unveiling Covert Harms and Social Threats in LLM Generated Conversationscited by 25
  28. 2024Exploring Multilingual Concepts of Human Value in Large Language Models: Is Value Alignment Consistent, Transferable and Controllable across Languages?cited by 24
  29. 2024DOSA: A Dataset of Social Artifacts from Different Indian Geographical Subculturescited by 24
  30. 2024CulturePark: Boosting Cross-cultural Understanding in Large Language Modelscited by 24
  31. 2024Extrinsic Evaluation of Cultural Competence in Large Language Modelscited by 24
  32. 2025Normative Evaluation of Large Language Models with Everyday Moral Dilemmascited by 24
  33. 2023CDEval: A Benchmark for Measuring the Cultural Dimensions of Large Language Modelscited by 23
  34. 2024Deconstructing The Ethics of Large Language Models from Long-standing Issues to New-emerging Dilemmas: A Surveycited by 23
  35. 2024CLAVE: An Adaptive Framework for Evaluating Values of LLM Generated Responsescited by 23
  36. 2025Multiple LLM Agents Debate for Equitable Cultural Alignmentcited by 21
  37. 2024The Potential and Challenges of Evaluating Attitudes, Opinions, and Values in Large Language Modelscited by 19
  38. 2024Self-Alignment: Improving Alignment of Cultural Values in LLMs via In-Context Learningcited by 19

50 most cited of 206 All 206 (JSON)