mlsanityData to 7 Oct 2026

Are Pretrained Multilingual Models Equally Fair Across Languages?

arXiv 2210.05457 ↗First posted 11 Oct 2022cs.CLcited by 8

Not on the map. Why?

Abstract (arXiv)

Pretrained multilingual language models can help bridge the digital language divide, enabling high-quality NLP models for lower resourced languages. Studies of multilingual models have so far focused on performance, consistency, and cross-lingual generalisation. However, with their wide-spread application in the wild and downstream societal impact, it is important to put multilingual models under the same scrutiny as monolingual models. This work investigates the group fairness of multilingual models, asking whether these models are equally fair across languages. To this end, we create a new four-way multilingual dataset of parallel cloze test examples (MozArt), equipped with demographic information (balanced with regard to gender and native tongue) about the test participants. We evaluate three multilingual models on MozArt -- mBERT, XLM-R, and mT5 -- and show that across the four target languages, the three models exhibit different levels of group disparity, e.g., exhibiting near-equal risk for Spanish, but high levels of disparity for German.

Key facts. Cited by 8 papers in the map (data through 7 Oct 2026). Builds on 18 of its 41 references that are in the map. First posted on arXiv 11 Oct 2022.

Builds on

18 of 41 references.

References · 18

  1. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,204
  2. 2019Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformercited by 14,491
  3. 2019Unsupervised Cross-lingual Representation Learning at Scalecited by 3,907
  4. 2019A Survey on Bias and Fairness in Machine Learningcited by 1,911
  5. 2020mT5: A massively multilingual pre-trained text-to-text transformercited by 1,813
  6. 2017Men Also Like Shopping: Reducing Gender Bias Amplification using Corpus-level Constraintscited by 420
  7. 2020Are All Languages Created Equal in Multilingual BERT?cited by 206
  8. 2018Reducing Gender Bias in Abusive Language Detectioncited by 152
  9. 2018Counterfactual Fairness in Text Classification through Robustnesscited by 150
  10. 2021The Perils of Using Mechanical Turk to Evaluate Open-Ended Text Generationcited by 96
  11. 2019Investigating Multilingual NMT Representations at Scalecited by 68
  12. 2020Gender Bias in Multilingual Embeddings and Cross-Lingual Transfercited by 55
6 more
  1. 2020Multilingual Twitter Corpus and Baselines for Evaluating Demographic Bias in Hate Speech Recognitioncited by 43
  2. 2021Assessing Multilingual Fairness in Pre-trained Multimodal Representationscited by 35
  3. 2020Probing Multilingual BERT for Genetic and Typological Signalscited by 26
  4. 2020What does it mean to be language-agnostic? Probing multilingual sentence encoders for typological propertiescited by 23
  5. 2021Attention Can Reflect Syntactic Structure (If You Let It)cited by 22
  6. 2020Type B Reflexivization as an Unambiguous Testbed for Multilingual Multi-Task Gender Biascited by 15

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2023: 3; 2024: 2; 2025: 2; 2026: 1.

8 papers.

  1. 2024A Survey on Multilingual Large Language Models: Corpora, Alignment, and Biascited by 82
  2. 2023Comparing Biases and the Impact of Multilingual Training across Multiple Languagescited by 36
  3. 2025Investigating Language Preference of Multilingual RAG Systemscited by 22
  4. 2023Multilingual large language models leak human stereotypes across language boundariescited by 12
  5. 2024Performance of Recent Large Language Models for a Low-Resourced Languagecited by 7
  6. 2023Towards a Deep Understanding of Multilingual End-to-End Speech Translationcited by 3
  7. 2025Bias Beyond English: Evaluating Social Bias and Debiasing Methods in a Low-Resource Settingcited by 1
  8. 2026Code-Switching Reveals Language Anchoring in Multilingual LLMscited by 1