mlsanityData to 7 Oct 2026

Multilingual Language Models are not Multicultural: A Case Study in Emotion

arXiv 2307.01370 ↗First posted 3 Jul 2023cs.CLcited by 54

Not on the map. Why?

Abstract (arXiv)

Emotions are experienced and expressed differently across the world. In order to use Large Language Models (LMs) for multilingual tasks that require emotional sensitivity, LMs must reflect this cultural variation in emotion. In this study, we investigate whether the widely-used multilingual LMs in 2023 reflect differences in emotional expressions across cultures and languages. We find that embeddings obtained from LMs (e.g., XLM-RoBERTa) are Anglocentric, and generative LMs (e.g., ChatGPT) reflect Western norms, even when responding to prompts in other languages. Our results show that multilingual LMs do not successfully learn the culturally appropriate nuances of emotion and we highlight possible research directions towards correcting this.

Key facts. Cited by 54 papers in the map (data through 7 Oct 2026). Builds on 12 of its 39 references that are in the map. First posted on arXiv 3 Jul 2023.

Builds on

12 of 39 references.

References · 12

  1. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,204
  2. 2019RoBERTa: A Robustly Optimized BERT Pretraining Approachcited by 13,266
  3. 2019Sentence-BERT: Sentence Embeddings using Siamese BERT-Networkscited by 9,049
  4. 2019Unsupervised Cross-lingual Representation Learning at Scalecited by 3,907
  5. 2020Making Monolingual Sentence Embeddings Multilingual using Knowledge Distillationcited by 591
  6. 2020Revisiting Pre-Trained Models for Chinese Natural Language Processingcited by 262
  7. 2023Assessing Cross-Cultural Alignment between ChatGPT and Human Societies: An Empirical Studycited by 203
  8. 2022Probing Pre-Trained Language Models for Cross-Cultural Differences in Valuescited by 147
  9. 2022BERTIN: Efficient Pre-Training of a Spanish Language Model using Perplexity Samplingcited by 30
  10. 2021An Evaluation of Generative Pre-Training Model-based Therapy Chatbot for Caregiverscited by 15
  11. 2020Learning and Evaluating Emotion Lexicons for 91 Languagescited by 13
  12. 2021Understanding Emails and Drafting Responses -- An Approach Using GPT-3cited by 10

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2023: 3; 2024: 24; 2025: 21; 2026: 6.

54 papers.

Sort:

By area:

  1. 2024Leak, Cheat, Repeat: Data Contamination and Evaluation Malpractices in Closed-Source LLMscited by 224
  2. 2024Towards Measuring and Modeling "Culture" in LLMs: A Surveycited by 166
  3. 2024The BiGGen Bench: A Principled Benchmark for Fine-grained Evaluation of Language Models with Language Modelscited by 89
  4. 2024Massively Multi-Cultural Knowledge Acquisition & LM Benchmarkingcited by 63
  5. 2025Culture is Not Trivia: Sociocultural Theory for Cultural NLPcited by 40
  6. 2025Multilingual != Multicultural: Evaluating Gaps Between Multilingual Capabilities and Cultural Alignment in LLMscited by 39
  7. 2024Evaluating Cultural Adaptability of a Large Language Model via Simulation of Synthetic Personascited by 36
  8. 2024KorNAT: LLM Alignment Benchmark for Korean Social Values and Common Knowledgecited by 33
  9. 2024Large Language Models Show Human-like Social Desirability Biases in Survey Responsescited by 27
  10. 2024CulturalBench: A Robust, Diverse, and Challenging Cultural Benchmark by Human-AI CulturalTeamingcited by 27
  11. 2025BRIGHTER: BRIdging the Gap in Human-Annotated Textual Emotion Recognition Datasets for 28 Languagescited by 27
  12. 2024Culturally Aware and Adapted NLP: A Taxonomy and a Survey of the State of the Artcited by 26
38 more
  1. 2025Break the Checkbox: Challenging Closed-Style Evaluations of Cultural Alignment in LLMscited by 26
  2. 2024BertaQA: How Much Do Language Models Know About Local Culture?cited by 23
  3. 2024Modeling Human Subjectivity in LLMs Using Explicit and Implicit Human Factors in Personascited by 23
  4. 2024High-Dimension Human Value Representation in Large Language Modelscited by 20
  5. 2026Large Language Model Reasoning Failurescited by 20
  6. 2025CULEMO: Cultural Lenses on Emotion -- Benchmarking LLMs for Cross-Cultural Emotion Understandingcited by 14
  7. 2024Vox Populi, Vox AI? Using Language Models to Estimate German Public Opinioncited by 13
  8. 2024Building Knowledge-Guided Lexica to Model Cultural Variationcited by 11
  9. 2024The Echoes of Multilinguality: Tracing Cultural Value Shifts during LM Fine-tuningcited by 10
  10. 2024mCSQA: Multilingual Commonsense Reasoning Dataset with Unified Creation Strategy by Language Models and Humanscited by 10
  11. 2025AI with Emotions: Exploring Emotional Expressions in Large Language Modelscited by 10
  12. 2025SemEval-2025 Task 11: Bridging the Gap in Text-Based Emotion Detectioncited by 9
  13. 2025KODIS: A Multicultural Dispute Resolution Dialogue Corpuscited by 9
  14. 2025Fluent but Foreign: Even Regional LLMs Lack Cultural Alignmentcited by 9
  15. 2024Are Generative Language Models Multicultural? A Study on Hausa Culture and Emotions using ChatGPTcited by 8
  16. 2024SOCIALITE-LLAMA: An Instruction-Tuned Model for Social Scientific Taskscited by 7
  17. 2024English Prompts are Better for NLI-based Zero-Shot Emotion Classification than Target-Language Promptscited by 7
  18. 2024Analyzing Cultural Representations of Emotions in LLMs through Mixed Emotion Surveycited by 7
  19. 2025CultureSynth: A Hierarchical Taxonomy-Guided and Retrieval-Augmented Framework for Cultural Question-Answer Synthesiscited by 6
  20. 2024I Am Not Them: Fluid Identities and Persistent Out-group Bias in Large Language Modelscited by 5
  21. 2024The FIX Benchmark: Extracting Features Interpretable to eXpertscited by 5
  22. 2025Is It Bad to Work All the Time? Cross-Cultural Evaluation of Social Norm Biases in GPT-4cited by 5
  23. 2025Culturally-Aware Conversations: A Framework & Benchmark for LLMscited by 5
  24. 2025Do Large Language Models Truly Understand Cross-cultural Differences?cited by 5
  25. 2025RedditESS: A Mental Health Social Support Interaction Dataset -- Understanding Effective Social Support to Refine AI-Driven Support Toolscited by 3
  26. 2023TorchQL: A Programming Framework for Integrity Constraints in Machine Learningcited by 2
  27. 2023Comparing Styles across Languages: A Cross-Cultural Exploration of Politenesscited by 2
  28. 2025Artificial Impressions: Evaluating Large Language Model Behavior Through the Lens of Trait Impressionscited by 2
  29. 2025Assessing Socio-Cultural Alignment and Technical Safety of Sovereign LLMscited by 2
  30. 2025T-FIX: Text-Based Explanations with Features Interpretable to eXpertscited by 2
  31. 2023OYXOY: A Modern NLP Test Suite for Modern Greekcited by 1
  32. 2024MASIVE: Open-Ended Affective State Identification in English and Spanishcited by 1
  33. 2024Locally Measuring Cross-lingual Lexical Alignment: A Domain and Word Level Perspectivecited by 1
  34. 2025LSC-Eval: A General Framework to Evaluate Methods for Assessing Dimensions of Lexical Semantic Change Using LLM-Generated Synthetic Datacited by 1
  35. 2025Exploring Cultural Nuances in Emotion Perception Across 15 African Languagescited by 1
  36. 2025Emotionally-Aware Agents for Dispute Resolutioncited by 1
  37. 2026Expressing Social Emotions: Misalignment Between LLMs and Human Cultural Emotion Normscited by 1
  38. 2025A Multifaceted Analysis of Social Biases in Large Language Modelscited by 0

50 most cited of 54 All 54 (JSON)