# Multilingual Language Models are not Multicultural: A Case Study in Emotion

arXiv: https://arxiv.org/abs/2307.01370 · HTML: https://mlsanity.com/paper/2307.01370/

## Key facts

- Cited by 54 papers in the map (data through 14 Oct 2026).
- Builds on 12 of its 39 references that are in the map.
- First posted on arXiv 3 Jul 2023.

## Builds on (most cited first, 12 of 12)

- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [RoBERTa: A Robustly Optimized BERT Pretraining Approach](https://mlsanity.com/paper/1907.11692/) (2019)
- [Sentence-BERT: Sentence Embeddings using Siamese BERT-Networks](https://mlsanity.com/paper/1908.10084/) (2019)
- [Unsupervised Cross-lingual Representation Learning at Scale](https://mlsanity.com/paper/1911.02116/) (2019)
- [Making Monolingual Sentence Embeddings Multilingual using Knowledge Distillation](https://mlsanity.com/paper/2004.09813/) (2020)
- [Revisiting Pre-Trained Models for Chinese Natural Language Processing](https://mlsanity.com/paper/2004.13922/) (2020)
- [Assessing Cross-Cultural Alignment between ChatGPT and Human Societies: An Empirical Study](https://mlsanity.com/paper/2303.17466/) (2023)
- [Probing Pre-Trained Language Models for Cross-Cultural Differences in Values](https://mlsanity.com/paper/2203.13722/) (2022)
- [BERTIN: Efficient Pre-Training of a Spanish Language Model using Perplexity Sampling](https://mlsanity.com/paper/2207.06814/) (2022)
- [An Evaluation of Generative Pre-Training Model-based Therapy Chatbot for Caregivers](https://mlsanity.com/paper/2107.13115/) (2021)
- [Learning and Evaluating Emotion Lexicons for 91 Languages](https://mlsanity.com/paper/2005.05672/) (2020)
- [Understanding Emails and Drafting Responses -- An Approach Using GPT-3](https://mlsanity.com/paper/2102.03062/) (2021)

## Built on by (most cited first, 30 of 54)

- [Leak, Cheat, Repeat: Data Contamination and Evaluation Malpractices in Closed-Source LLMs](https://mlsanity.com/paper/2402.03927/) (2024)
- [Towards Measuring and Modeling "Culture" in LLMs: A Survey](https://mlsanity.com/paper/2403.15412/) (2024)
- [The BiGGen Bench: A Principled Benchmark for Fine-grained Evaluation of Language Models with Language Models](https://mlsanity.com/paper/2406.05761/) (2024)
- [Massively Multi-Cultural Knowledge Acquisition & LM Benchmarking](https://mlsanity.com/paper/2402.09369/) (2024)
- [Culture is Not Trivia: Sociocultural Theory for Cultural NLP](https://mlsanity.com/paper/2502.12057/) (2025)
- [Multilingual != Multicultural: Evaluating Gaps Between Multilingual Capabilities and Cultural Alignment in LLMs](https://mlsanity.com/paper/2502.16534/) (2025)
- [Evaluating Cultural Adaptability of a Large Language Model via Simulation of Synthetic Personas](https://mlsanity.com/paper/2408.06929/) (2024)
- [KorNAT: LLM Alignment Benchmark for Korean Social Values and Common Knowledge](https://mlsanity.com/paper/2402.13605/) (2024)
- [CulturalBench: A Robust, Diverse, and Challenging Cultural Benchmark by Human-AI CulturalTeaming](https://mlsanity.com/paper/2410.02677/) (2024)
- [Large Language Models Show Human-like Social Desirability Biases in Survey Responses](https://mlsanity.com/paper/2405.06058/) (2024)
- [BRIGHTER: BRIdging the Gap in Human-Annotated Textual Emotion Recognition Datasets for 28 Languages](https://mlsanity.com/paper/2502.11926/) (2025)
- [Culturally Aware and Adapted NLP: A Taxonomy and a Survey of the State of the Art](https://mlsanity.com/paper/2406.03930/) (2024)
- [Break the Checkbox: Challenging Closed-Style Evaluations of Cultural Alignment in LLMs](https://mlsanity.com/paper/2502.08045/) (2025)
- [BertaQA: How Much Do Language Models Know About Local Culture?](https://mlsanity.com/paper/2406.07302/) (2024)
- [Modeling Human Subjectivity in LLMs Using Explicit and Implicit Human Factors in Personas](https://mlsanity.com/paper/2406.14462/) (2024)
- [Large Language Model Reasoning Failures](https://mlsanity.com/paper/2602.06176/) (2026)
- [High-Dimension Human Value Representation in Large Language Models](https://mlsanity.com/paper/2404.07900/) (2024)
- [CULEMO: Cultural Lenses on Emotion -- Benchmarking LLMs for Cross-Cultural Emotion Understanding](https://mlsanity.com/paper/2503.10688/) (2025)
- [Vox Populi, Vox AI? Using Language Models to Estimate German Public Opinion](https://mlsanity.com/paper/2407.08563/) (2024)
- [Building Knowledge-Guided Lexica to Model Cultural Variation](https://mlsanity.com/paper/2406.11622/) (2024)
- [AI with Emotions: Exploring Emotional Expressions in Large Language Models](https://mlsanity.com/paper/2504.14706/) (2025)
- [The Echoes of Multilinguality: Tracing Cultural Value Shifts during LM Fine-tuning](https://mlsanity.com/paper/2405.12744/) (2024)
- [mCSQA: Multilingual Commonsense Reasoning Dataset with Unified Creation Strategy by Language Models and Humans](https://mlsanity.com/paper/2406.04215/) (2024)
- [Fluent but Foreign: Even Regional LLMs Lack Cultural Alignment](https://mlsanity.com/paper/2505.21548/) (2025)
- [SemEval-2025 Task 11: Bridging the Gap in Text-Based Emotion Detection](https://mlsanity.com/paper/2503.07269/) (2025)
- [KODIS: A Multicultural Dispute Resolution Dialogue Corpus](https://mlsanity.com/paper/2504.12723/) (2025)
- [Are Generative Language Models Multicultural? A Study on Hausa Culture and Emotions using ChatGPT](https://mlsanity.com/paper/2406.19504/) (2024)
- [SOCIALITE-LLAMA: An Instruction-Tuned Model for Social Scientific Tasks](https://mlsanity.com/paper/2402.01980/) (2024)
- [English Prompts are Better for NLI-based Zero-Shot Emotion Classification than Target-Language Prompts](https://mlsanity.com/paper/2402.03223/) (2024)
- [Analyzing Cultural Representations of Emotions in LLMs through Mixed Emotion Survey](https://mlsanity.com/paper/2408.02143/) (2024)

## Abstract (arXiv)

Emotions are experienced and expressed differently across the world. In order to use Large Language Models (LMs) for multilingual tasks that require emotional sensitivity, LMs must reflect this cultural variation in emotion. In this study, we investigate whether the widely-used multilingual LMs in 2023 reflect differences in emotional expressions across cultures and languages. We find that embeddings obtained from LMs (e.g., XLM-RoBERTa) are Anglocentric, and generative LMs (e.g., ChatGPT) reflect Western norms, even when responding to prompts in other languages. Our results show that multilingual LMs do not successfully learn the culturally appropriate nuances of emotion and we highlight possible research directions towards correcting this.