mlsanityData to 7 Oct 2026

Investigating Bias in Multilingual Language Models: Cross-Lingual Transfer of Debiasing Techniques

arXiv 2310.10310 ↗First posted 16 Oct 2023cs.CLcited by 14

Not on the map. Why?

Abstract (arXiv)

This paper investigates the transferability of debiasing techniques across different languages within multilingual models. We examine the applicability of these techniques in English, French, German, and Dutch. Using multilingual BERT (mBERT), we demonstrate that cross-lingual transfer of debiasing techniques is not only feasible but also yields promising results. Surprisingly, our findings reveal no performance disadvantages when applying these techniques to non-English languages. Using translations of the CrowS-Pairs dataset, our analysis identifies SentenceDebias as the best technique across different languages, reducing bias in mBERT by an average of 13%. We also find that debiasing techniques with additional pretraining exhibit enhanced cross-lingual effectiveness for the languages included in the analyses, particularly in lower-resource languages. These novel insights contribute to a deeper understanding of bias mitigation in multilingual language models and provide practical guidance for debiasing techniques in different language contexts.

Key facts. Cited by 14 papers in the map (data through 7 Oct 2026). Builds on 20 of its 24 references that are in the map. First posted on arXiv 16 Oct 2023.

Builds on

20 of 24 references.

References · 20

  1. 2016Man is to Computer Programmer as Woman is to Homemaker? Debiasing Word Embeddingscited by 1,304
  2. 2020CrowS-Pairs: A Challenge Dataset for Measuring Social Biases in Masked Language Modelscited by 703
  3. 2018Gender Bias in Coreference Resolution: Evaluation and Debiasing Methodscited by 657
  4. 2019How multilingual is Multilingual BERT?cited by 642
  5. 2020Null It Out: Guarding Protected Attributes by Iterative Nullspace Projectioncited by 442
  6. 2019Mitigating Gender Bias in Natural Language Processing: Literature Reviewcited by 287
  7. 2020Measuring and Reducing Gendered Correlations in Pre-trained Modelscited by 222
  8. 2020Are All Languages Created Equal in Multilingual BERT?cited by 206
  9. 2019Gender Bias in Contextualized Word Embeddingscited by 203
  10. 2019Counterfactual Data Augmentation for Mitigating Gender Stereotypes in Languages with Rich Morphologycited by 200
  11. 2020Towards Debiasing Sentence Representationscited by 170
  12. 2018Learning Gender-Neutral Word Embeddingscited by 169
8 more
  1. 2021Sustainable Modular Debiasing of Language Modelscited by 98
  2. 2021An Empirical Survey of the Effectiveness of Debiasing Techniques for Pre-trained Language Modelscited by 96
  3. 2021A Survey on Gender Bias in Natural Language Processingcited by 89
  4. 2021Mitigating Language-Dependent Ethnic Bias in BERTcited by 87
  5. 2020Gender Bias in Multilingual Embeddings and Cross-Lingual Transfercited by 55
  6. 2019How Language-Neutral is Multilingual BERT?cited by 51
  7. 2023Comparing Biases and the Impact of Multilingual Training across Multiple Languagescited by 36
  8. 2019Analytical Methods for Interpretable Ultradense Word Embeddingscited by 13

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2024: 5; 2025: 6; 2026: 3.

14 papers.

Sort:

By area:

  1. 2025The State of Multilingual LLM Safety Research: From Measuring the Language Gap to Mitigating Itcited by 42
  2. 2024MBBQ: A Dataset for Cross-Lingual Comparison of Stereotypes in Generative LLMscited by 32
  3. 2024Teaching LLMs to Abstain across Languages via Multilingual Feedbackcited by 14
  4. 2024Cross-Lingual Transfer of Debiasing and Detoxification in Multilingual LLMs: An Extensive Investigationcited by 5
  5. 2024Evaluating Short-Term Temporal Fluctuations of Social Biases in Social Media Data and Masked Language Modelscited by 3
  6. 2024Pruning Multilingual Large Language Models for Multilingual Inferencecited by 2
  7. 2025Bias Attribution in Filipino Language Models: Extending a Bias Interpretability Metric for Application on Agglutinative Languagescited by 2
  8. 2025Debiasing Multilingual LLMs in Cross-lingual Latent Spacecited by 2
  9. 2026Language-Coupled Reinforcement Learning for Multilingual Retrieval-Augmented Generationcited by 2
  10. 2025Bias Beyond English: Evaluating Social Bias and Debiasing Methods in a Low-Resource Settingcited by 1
  11. 2025Iterative Multilingual Spectral Attribute Erasurecited by 1
  12. 2025PBBQ: A Persian Bias Benchmark Dataset Curated with Human-AI Collaboration for Large Language Modelscited by 1
2 more
  1. 2026Scalable and Culturally Specific Stereotype Dataset Construction via Human-LLM Collaborationcited by 1
  2. 2026Multiple-Debias: A Full-process Debiasing Method for Multilingual Pre-trained Language Modelscited by 0