# Investigating Bias in Multilingual Language Models: Cross-Lingual Transfer of Debiasing Techniques

arXiv: https://arxiv.org/abs/2310.10310 · HTML: https://mlsanity.com/paper/2310.10310/

## Key facts

- Cited by 14 papers in the map (data through 14 Oct 2026).
- Builds on 20 of its 24 references that are in the map.
- First posted on arXiv 16 Oct 2023.

## Builds on (most cited first, 20 of 20)

- [Man is to Computer Programmer as Woman is to Homemaker? Debiasing Word Embeddings](https://mlsanity.com/paper/1607.06520/) (2016)
- [CrowS-Pairs: A Challenge Dataset for Measuring Social Biases in Masked Language Models](https://mlsanity.com/paper/2010.00133/) (2020)
- [Gender Bias in Coreference Resolution: Evaluation and Debiasing Methods](https://mlsanity.com/paper/1804.06876/) (2018)
- [How multilingual is Multilingual BERT?](https://mlsanity.com/paper/1906.01502/) (2019)
- [Null It Out: Guarding Protected Attributes by Iterative Nullspace Projection](https://mlsanity.com/paper/2004.07667/) (2020)
- [Mitigating Gender Bias in Natural Language Processing: Literature Review](https://mlsanity.com/paper/1906.08976/) (2019)
- [Measuring and Reducing Gendered Correlations in Pre-trained Models](https://mlsanity.com/paper/2010.06032/) (2020)
- [Are All Languages Created Equal in Multilingual BERT?](https://mlsanity.com/paper/2005.09093/) (2020)
- [Gender Bias in Contextualized Word Embeddings](https://mlsanity.com/paper/1904.03310/) (2019)
- [Counterfactual Data Augmentation for Mitigating Gender Stereotypes in Languages with Rich Morphology](https://mlsanity.com/paper/1906.04571/) (2019)
- [Towards Debiasing Sentence Representations](https://mlsanity.com/paper/2007.08100/) (2020)
- [Learning Gender-Neutral Word Embeddings](https://mlsanity.com/paper/1809.01496/) (2018)
- [Sustainable Modular Debiasing of Language Models](https://mlsanity.com/paper/2109.03646/) (2021)
- [An Empirical Survey of the Effectiveness of Debiasing Techniques for Pre-trained Language Models](https://mlsanity.com/paper/2110.08527/) (2021)
- [A Survey on Gender Bias in Natural Language Processing](https://mlsanity.com/paper/2112.14168/) (2021)
- [Mitigating Language-Dependent Ethnic Bias in BERT](https://mlsanity.com/paper/2109.05704/) (2021)
- [Gender Bias in Multilingual Embeddings and Cross-Lingual Transfer](https://mlsanity.com/paper/2005.00699/) (2020)
- [How Language-Neutral is Multilingual BERT?](https://mlsanity.com/paper/1911.03310/) (2019)
- [Comparing Biases and the Impact of Multilingual Training across Multiple Languages](https://mlsanity.com/paper/2305.11242/) (2023)
- [Analytical Methods for Interpretable Ultradense Word Embeddings](https://mlsanity.com/paper/1904.08654/) (2019)

## Built on by (most cited first, 14 of 14)

- [The State of Multilingual LLM Safety Research: From Measuring the Language Gap to Mitigating It](https://mlsanity.com/paper/2505.24119/) (2025)
- [MBBQ: A Dataset for Cross-Lingual Comparison of Stereotypes in Generative LLMs](https://mlsanity.com/paper/2406.07243/) (2024)
- [Teaching LLMs to Abstain across Languages via Multilingual Feedback](https://mlsanity.com/paper/2406.15948/) (2024)
- [Cross-Lingual Transfer of Debiasing and Detoxification in Multilingual LLMs: An Extensive Investigation](https://mlsanity.com/paper/2412.14050/) (2024)
- [Evaluating Short-Term Temporal Fluctuations of Social Biases in Social Media Data and Masked Language Models](https://mlsanity.com/paper/2406.13556/) (2024)
- [Pruning Multilingual Large Language Models for Multilingual Inference](https://mlsanity.com/paper/2409.16911/) (2024)
- [Bias Attribution in Filipino Language Models: Extending a Bias Interpretability Metric for Application on Agglutinative Languages](https://mlsanity.com/paper/2506.07249/) (2025)
- [Debiasing Multilingual LLMs in Cross-lingual Latent Space](https://mlsanity.com/paper/2508.17948/) (2025)
- [Language-Coupled Reinforcement Learning for Multilingual Retrieval-Augmented Generation](https://mlsanity.com/paper/2601.14896/) (2026)
- [Bias Beyond English: Evaluating Social Bias and Debiasing Methods in a Low-Resource Setting](https://mlsanity.com/paper/2504.11183/) (2025)
- [Iterative Multilingual Spectral Attribute Erasure](https://mlsanity.com/paper/2506.11244/) (2025)
- [PBBQ: A Persian Bias Benchmark Dataset Curated with Human-AI Collaboration for Large Language Models](https://mlsanity.com/paper/2510.19616/) (2025)
- [Scalable and Culturally Specific Stereotype Dataset Construction via Human-LLM Collaboration](https://mlsanity.com/paper/2607.07895/) (2026)
- [Multiple-Debias: A Full-process Debiasing Method for Multilingual Pre-trained Language Models](https://mlsanity.com/paper/2604.02772/) (2026)

## Abstract (arXiv)

This paper investigates the transferability of debiasing techniques across different languages within multilingual models. We examine the applicability of these techniques in English, French, German, and Dutch. Using multilingual BERT (mBERT), we demonstrate that cross-lingual transfer of debiasing techniques is not only feasible but also yields promising results. Surprisingly, our findings reveal no performance disadvantages when applying these techniques to non-English languages. Using translations of the CrowS-Pairs dataset, our analysis identifies SentenceDebias as the best technique across different languages, reducing bias in mBERT by an average of 13%. We also find that debiasing techniques with additional pretraining exhibit enhanced cross-lingual effectiveness for the languages included in the analyses, particularly in lower-resource languages. These novel insights contribute to a deeper understanding of bias mitigation in multilingual language models and provide practical guidance for debiasing techniques in different language contexts.