mlsanityData to 14 Oct 2026

Multilingual large language models leak human stereotypes across language boundaries

arXiv 2312.07141 ↗First posted 12 Dec 2023cs.CLcited by 12

Not on the map. Why?

Abstract (arXiv)

Multilingual large language models have gained prominence for their proficiency in processing and generating text across languages. Like their monolingual counterparts, multilingual models are likely to pick up on stereotypes and other social biases present in their training data. In this paper, we study a phenomenon we term stereotype leakage, which refers to how training a model multilingually may lead to stereotypes expressed in one language showing up in the models' behaviour in another. We propose a measurement framework for stereotype leakage and investigate its effect across English, Russian, Chinese, and Hindi and with GPT-3.5, mT5, and mBERT. Our findings show a noticeable leakage of positive, negative, and non-polar associations across all languages. We find that of these models, GPT-3.5 exhibits the most stereotype leakage, and Hindi is the most susceptible to leakage effects. WARNING: This paper contains model outputs which could be offensive in nature.

Key facts. Cited by 12 papers in the map (data through 14 Oct 2026). Builds on 24 of its 54 references that are in the map. First posted on arXiv 12 Dec 2023.

Builds on

24 of 54 references.

References · 24

  1. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,399
  2. 2020Language Models are Few-Shot Learnerscited by 32,419
  3. 2022Training language models to follow instructions with human feedbackcited by 15,971
  4. 2020mT5: A massively multilingual pre-trained text-to-text transformercited by 1,818
  5. 2019Cross-lingual Language Model Pretrainingcited by 1,090
  6. 2021Ethical and social risks of harm from Language Modelscited by 966
  7. 2020StereoSet: Measuring stereotypical bias in pretrained language modelscited by 924
  8. 2020The State and Fate of Linguistic Diversity and Inclusion in the NLP Worldcited by 798
  9. 2020CrowS-Pairs: A Challenge Dataset for Measuring Social Biases in Masked Language Modelscited by 710
  10. 2022Language Models are Multilingual Chain-of-Thought Reasonerscited by 530
  11. 2021Few-shot Learning with Multilingual Language Modelscited by 278
  12. 2019Measuring Bias in Contextualized Word Representationscited by 276
12 more
  1. 2019Cross-Lingual Ability of Multilingual BERT: An Empirical Studycited by 182
  2. 2023Generative Language Models and Automated Influence Operations: Emerging Threats and Potential Mitigationscited by 158
  3. 2020When Being Unseen from mBERT is just the Beginning: Handling New Languages With Multilingual Language Modelscited by 131
  4. 2020Unmasking Contextual Stereotypes: Measuring and Mitigating BERT's Gender Biascited by 82
  5. 2022Gender Bias in Masked Language Models for Multiple Languagescited by 60
  6. 2020Gender Bias in Multilingual Embeddings and Cross-Lingual Transfercited by 55
  7. 2023Comparing Biases and the Impact of Multilingual Training across Multiple Languagescited by 36
  8. 2021Assessing Multilingual Fairness in Pre-trained Multimodal Representationscited by 35
  9. 2022Theory-Grounded Measurement of U.S. Social Stereotypes in English Language Modelscited by 35
  10. 2020Type B Reflexivization as an Unambiguous Testbed for Multilingual Multi-Task Gender Biascited by 15
  11. 2022Mapping the Multilingual Margins: Intersectional Biases of Sentiment Analysis Systems in English, Spanish, and Arabiccited by 13
  12. 2022Are Pretrained Multilingual Models Equally Fair Across Languages?cited by 8

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2024: 6; 2025: 5; 2026: 1.

12 papers.

  1. 2024Unintended Impacts of LLM Alignment on Global Representationcited by 66
  2. 2024MultiTrust: A Comprehensive Benchmark Towards Trustworthy Multimodal Large Language Modelscited by 53
  3. 2024Multilingual Large Language Model: A Survey of Resources, Taxonomy and Frontierscited by 51
  4. 2024Evaluating Cultural Adaptability of a Large Language Model via Simulation of Synthetic Personascited by 37
  5. 2024Multi-FAct: Assessing Factuality of Multilingual LLMs using FActScorecited by 18
  6. 2024BiasDora: Exploring Hidden Biased Associations in Vision-Language Modelscited by 12
  7. 2025FairI Tales: Evaluation of Fairness in Indian Contexts with a Focus on Bias and Stereotypescited by 9
  8. 2025Language Models Predict Empathy Gaps Between Social In-groups and Out-groupscited by 6
  9. 2026Failure of contextual invariance in large language modelscited by 3
  10. 2025Losing our Tail, Again: (Un)Natural Selection & Multilingual LLMscited by 1
  11. 2025Am I Blue or Is My Hobby Counting Teardrops? Expression Leakage in Large Language Models as a Symptom of Irrelevancy Disruptioncited by 1
  12. 2025Is Lying Only Sinful in Islam? Exploring Religious Bias in Multilingual Large Language Models Across Major Religionscited by 1