mlsanityData to 7 Oct 2026

On Evaluating and Mitigating Gender Biases in Multilingual Settings

arXiv 2307.01503 ↗First posted 4 Jul 2023cs.CLcited by 31

Not on the map. Why?

Abstract (arXiv)

While understanding and removing gender biases in language models has been a long-standing problem in Natural Language Processing, prior research work has primarily been limited to English. In this work, we investigate some of the challenges with evaluating and mitigating biases in multilingual settings which stem from a lack of existing benchmarks and resources for bias evaluation beyond English especially for non-western context. In this paper, we first create a benchmark for evaluating gender biases in pre-trained masked language models by extending DisCo to different Indian languages using human annotations. We extend various debiasing methods to work beyond English and evaluate their effectiveness for SOTA massively multilingual models on our proposed metric. Overall, our work highlights the challenges that arise while studying social biases in multilingual settings and provides resources as well as mitigation techniques to take a step toward scaling to more languages.

Key facts. Cited by 31 papers in the map (data through 7 Oct 2026). Builds on 31 of its 42 references that are in the map. First posted on arXiv 4 Jul 2023.

Builds on

31 of 42 references.

References · 31

  1. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,204
  2. 2020Language Models are Few-Shot Learnerscited by 32,199
  3. 2019Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformercited by 14,491
  4. 2019Parameter-Efficient Transfer Learning for NLPcited by 4,195
  5. 2019Unsupervised Cross-lingual Representation Learning at Scalecited by 3,907
  6. 2020mT5: A massively multilingual pre-trained text-to-text transformercited by 1,813
  7. 2019Energy and Policy Considerations for Deep Learning in NLPcited by 1,242
  8. 2016ConceptNet 5.5: An Open Multilingual Graph of General Knowledgecited by 1,065
  9. 2020StereoSet: Measuring stereotypical bias in pretrained language modelscited by 911
  10. 2020The State and Fate of Linguistic Diversity and Inclusion in the NLP Worldcited by 789
  11. 2018Gender Bias in Coreference Resolution: Evaluation and Debiasing Methodscited by 657
  12. 2019The Woman Worked as a Babysitter: On Biases in Language Generationcited by 540
19 more
  1. 2020AraBERT: Transformer-based Model for Arabic Language Understandingcited by 359
  2. 2021Self-Diagnosis and Self-Debiasing: A Proposal for Reducing Corpus-Based Bias in NLPcited by 336
  3. 2019Measuring Bias in Contextualized Word Representationscited by 275
  4. 2020Revisiting Pre-Trained Models for Chinese Natural Language Processingcited by 262
  5. 2020Measuring and Reducing Gendered Correlations in Pre-trained Modelscited by 222
  6. 2021Polyjuice: Generating Counterfactuals for Explaining, Evaluating, and Improving Modelscited by 219
  7. 2019Counterfactual Data Augmentation for Mitigating Gender Stereotypes in Languages with Rich Morphologycited by 200
  8. 2020Social Biases in NLP Models as Barriers for Persons with Disabilitiescited by 182
  9. 2020German's Next Language Modelcited by 144
  10. 2021Re-imagining Algorithmic Fairness in India and Beyondcited by 102
  11. 2021Sustainable Modular Debiasing of Language Modelscited by 98
  12. 2021An Empirical Survey of the Effectiveness of Debiasing Techniques for Pre-trained Language Modelscited by 96
  13. 2022Language Contamination Helps Explain the Cross-lingual Capabilities of English Pretrained Modelscited by 90
  14. 2021Mitigating Language-Dependent Ethnic Bias in BERTcited by 87
  15. 2022Gender Bias in Masked Language Models for Multiple Languagescited by 60
  16. 2020Gender Bias in Multilingual Embeddings and Cross-Lingual Transfercited by 55
  17. 2022Re-contextualizing Fairness in NLP: The Case of Indiacited by 55
  18. 2022Text Generation with Text-Editing Modelscited by 21
  19. 2021Debiasing Multilingual Word Embeddings: A Case Study of Three Indian Languagescited by 8

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2023: 1; 2024: 15; 2025: 12; 2026: 3.

31 papers.

Sort:

By area:

  1. 2024Aya 23: Open Weight Releases to Further Multilingual Progresscited by 121
  2. 2024Aya Expanse: Combining Research Breakthroughs for a New Multilingual Frontiercited by 121
  3. 2024The Multilingual Alignment Prism: Aligning Global and Local Preferences to Reduce Harmcited by 67
  4. 2024RLHF Can Speak Many Languages: Unlocking Multilingual Preference Optimization for LLMscited by 42
  5. 2024How Does Quantization Affect Multilingual LLMs?cited by 41
  6. 2024MBBQ: A Dataset for Cross-Lingual Comparison of Stereotypes in Generative LLMscited by 32
  7. 2024SeeGULL Multilingual: a Dataset of Geo-Culturally Situated Stereotypescited by 24
  8. 2025BiasEdit: Debiasing Stereotyped Language Models via Model Editingcited by 18
  9. 2023Indian-BhED: A Dataset for Measuring India-Centric Biases in Large Language Modelscited by 17
  10. 2024From 'Showgirls' to 'Performers': Fine-tuning with Gender-inclusive Language for Bias Reduction in LLMscited by 15
  11. 2024Social Bias in Large Language Models For Bangla: An Empirical Study on Gender and Religious Biascited by 13
  12. 2024Do Multilingual Large Language Models Mitigate Stereotype Bias?cited by 13
19 more
  1. 2025Executable Counterfactuals: Improving LLMs' Causal Reasoning Through Codecited by 10
  2. 2024GenderCARE: A Comprehensive Framework for Assessing and Reducing Gender Bias in Large Language Modelscited by 9
  3. 2025The Hidden Space of Safety: Understanding Preference-Tuned LLMs in Multilingual contextcited by 6
  4. 2025Bhaasha, Bhasa, Zaban: A Survey for Low-Resourced Languages in South Asia -- Current Stage and Challengescited by 6
  5. 2024Cross-Lingual Transfer of Debiasing and Detoxification in Multilingual LLMs: An Extensive Investigationcited by 5
  6. 2024Different Bias Under Different Criteria: Assessing Bias in LLMs with a Fact-Based Approachcited by 3
  7. 2025Exploring Gender Bias in Large Language Models: An In-depth Dive into the German Languagecited by 3
  8. 2026The Script Tax: Measuring Tokenization-Driven Efficiency and Latency Disparities in Multilingual Language Modelscited by 3
  9. 2025CONGRAD:Conflicting Gradient Filtering for Multilingual Preference Alignmentcited by 2
  10. 2025Camellia: Benchmarking Cultural Biases in LLMs for Asian Languagescited by 2
  11. 2024Beyond Generative Artificial Intelligence: Roadmap for Natural Language Generationcited by 1
  12. 2024Current State-of-the-Art of Bias Detection and Mitigation in Machine Translation for African and European Languages: a Reviewcited by 1
  13. 2025Iterative Multilingual Spectral Attribute Erasurecited by 1
  14. 2025Social Bias in Multilingual Language Models: A Surveycited by 1
  15. 2025IndiCASA: A Dataset and Bias Evaluation Framework in LLMs Using Contrastive Embedding Similarity in the Indian Contextcited by 1
  16. 2026Scalable and Culturally Specific Stereotype Dataset Construction via Human-LLM Collaborationcited by 1
  17. 2025Assessing Agentic Large Language Models in Multilingual National Biascited by 0
  18. 2025Teaching and Critiquing Conceptualization and Operationalization in NLPcited by 0
  19. 2026Multiple-Debias: A Full-process Debiasing Method for Multilingual Pre-trained Language Modelscited by 0