mlsanityData to 7 Oct 2026

Multiple-Debias: A Full-process Debiasing Method for Multilingual Pre-trained Language Models

arXiv 2604.02772 ↗First posted 3 Apr 2026cs.CLcited by 0

Not on the map. Why?

Abstract (arXiv)

Multilingual Pre-trained Language Models (MPLMs) have become essential tools for natural language processing. However, they often exhibit biases related to sensitive attributes such as gender, race, and religion. In this paper, we introduce a comprehensive multilingual debiasing method named Multiple-Debias to address these issues across multiple languages. By incorporating multilingual counterfactual data augmentation and multilingual Self-Debias across both pre-processing and post-processing stages, alongside parameter-efficient fine-tuning, we significantly reduced biases in MPLMs across three sensitive attributes in four languages. We also extended CrowS-Pairs to German, Spanish, Chinese, and Japanese, validating our full-process multilingual debiasing method for gender, racial, and religious bias. Our experiments show that (i) multilingual debiasing methods surpass monolingual approaches in effectively mitigating biases, and (ii) integrating debiasing information from different languages notably improves the fairness of MPLMs.

Key facts. No citing papers in the map yet (data through 7 Oct 2026). Builds on 20 of its 23 references that are in the map. First posted on arXiv 3 Apr 2026.

Builds on

20 of 23 references.

References · 20

  1. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,204
  2. 2020Language Models are Few-Shot Learnerscited by 32,199
  3. 2023LLaMA: Open and Efficient Foundation Language Modelscited by 13,958
  4. 2019Parameter-Efficient Transfer Learning for NLPcited by 4,195
  5. 2019Unsupervised Cross-lingual Representation Learning at Scalecited by 3,907
  6. 2021Prefix-Tuning: Optimizing Continuous Prompts for Generationcited by 3,878
  7. 2021The Power of Scale for Parameter-Efficient Prompt Tuningcited by 3,769
  8. 2020CrowS-Pairs: A Challenge Dataset for Measuring Social Biases in Masked Language Modelscited by 703
  9. 2018Gender Bias in Coreference Resolution: Evaluation and Debiasing Methodscited by 657
  10. 2021Self-Diagnosis and Self-Debiasing: A Proposal for Reducing Corpus-Based Bias in NLPcited by 336
  11. 2020Social Biases in NLP Models as Barriers for Persons with Disabilitiescited by 182
  12. 2021An Empirical Survey of the Effectiveness of Debiasing Techniques for Pre-trained Language Modelscited by 96
8 more
  1. 2021Mitigating Language-Dependent Ethnic Bias in BERTcited by 87
  2. 2024A Survey on Multilingual Large Language Models: Corpora, Alignment, and Biascited by 82
  3. 2022Gender Bias in Masked Language Models for Multiple Languagescited by 60
  4. 2020Gender Bias in Multilingual Embeddings and Cross-Lingual Transfercited by 55
  5. 2023On Evaluating and Mitigating Gender Biases in Multilingual Settingscited by 31
  6. 2023Fairness in Language Models Beyond English: Gaps and Challengescited by 16
  7. 2023An Empirical Analysis of Parameter-Efficient Methods for Debiasing Pre-Trained Language Modelscited by 16
  8. 2023Investigating Bias in Multilingual Language Models: Cross-Lingual Transfer of Debiasing Techniquescited by 14

Built on by

Not cited yet.