Data to 7 Oct 2026

Reducing Gender Bias in Word-Level Language Models with a Gender-Equalizing Loss Function

arXiv 1905.12801 ↗First posted 30 May 2019cs.CLcited by 57

Not on the map. Why?

Abstract (arXiv)

Gender bias exists in natural language datasets which neural language models tend to learn, resulting in biased text generation. In this research, we propose a debiasing approach based on the loss function modification. We introduce a new term to the loss function which attempts to equalize the probabilities of male and female words in the output. Using an array of bias evaluation metrics, we provide empirical evidence that our approach successfully mitigates gender bias in language models without increasing perplexity. In comparison to existing debiasing strategies, data augmentation, and word embedding debiasing, our method performs better in several aspects, especially in reducing gender bias in occupation words. Finally, we introduce a combination of data augmentation and our approach, and show that it outperforms existing strategies in all bias evaluation metrics.

Key facts. Cited by 57 papers in the map (data through 7 Oct 2026). First posted on arXiv 30 May 2019.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2021: 11; 2022: 12; 2023: 15; 2024: 13; 2025: 5; 2026: 1.

57 papers.

Sort:

By area:

2022LaMDA: Language Models for Dialog Applicationscited by 1,218
2023Bias and Fairness in Large Language Models: A Surveycited by 626
2021Persistent Anti-Muslim Bias in Large Language Modelscited by 376
2021Towards Understanding and Mitigating Social Biases in Language Modelscited by 294
2021Societal Biases in Language Generation: Progress and Challengescited by 201
2021RedditBias: A Real-World Resource for Bias Evaluation and Debiasing of Conversational Language Modelscited by 149
2022A Survey of Controllable Text Generation using Transformer-based Pre-trained Language Modelscited by 128
2021Sustainable Modular Debiasing of Language Modelscited by 98
2024Bias and Unfairness in Information Retrieval Systems: New Challenges in the LLM Eracited by 97
2021A Survey on Gender Bias in Natural Language Processingcited by 89
2024Bias in Large Language Models: Origin, Evaluation, and Mitigationcited by 77
2022Theories of "Gender" in NLP Bias Researchcited by 62

38 more

2024Self-Debiasing Large Language Models: Zero-Shot Recognition and Reduction of Stereotypescited by 58
2021Directional Bias Amplificationcited by 54
2023Identifying and Mitigating Vulnerabilities in LLM-Integrated Applicationscited by 54
2023Are Personalized Stochastic Parrots More Dangerous? Evaluating Persona Biases in Dialogue Systemscited by 37
2021Improving Gender Fairness of Pre-Trained Language Models without Catastrophic Forgettingcited by 34
2022Unified Detoxifying and Debiasing in Language Generation via Inference-time Adaptive Optimizationcited by 32
2025Actions Speak Louder than Words: Agent Decisions Reveal Implicit Biases in Language Modelscited by 30
2022Choose Your Lenses: Flaws in Gender Bias Evaluationcited by 29
2023Sociodemographic Bias in Language Models: A Survey and Forward Pathcited by 27
2023CHBias: Bias Evaluation and Mitigation of Chinese Conversational Language Modelscited by 25
2024Deconstructing The Ethics of Large Language Models from Long-standing Issues to New-emerging Dilemmas: A Surveycited by 23
2022Debiasing Methods for Fairer Neural Models in Vision and Language Research: A Surveycited by 22
2024Walking in Others' Shoes: How Perspective-Taking Guides Large Language Models in Reducing Toxicity and Biascited by 17
2022Don't Forget About Pronouns: Removing Gender Bias in Language Models Without Losing Factual Gender Informationcited by 16
2021Invariant Language Modelingcited by 15
2023Safety and Fairness for Content Moderation in Generative Modelscited by 15
2024Take Care of Your Prompt Bias! Investigating and Mitigating Prompt Bias in Factual Knowledge Extractioncited by 13
2023Tackling Bias in Pre-trained Language Models: Current Trends and Under-represented Societiescited by 11
2023Towards Coding Social Science Datasets with Language Modelscited by 10
2024Are Large Language Models Really Bias-Free? Jailbreak Prompts for Assessing Adversarial Robustness to Bias Elicitationcited by 9
2022Towards an Enhanced Understanding of Bias in Pre-trained Neural Language Models: A Survey with Special Emphasis on Affective Biascited by 8
2023FineDeb: A Debiasing Framework for Language Modelscited by 7
2023Debiasing should be Good and Bad: Measuring the Consistency of Debiasing Techniques in Language Modelscited by 7
2021DebIE: A Platform for Implicit and Explicit Debiasing of Word Embedding Spacescited by 5
2022Bridging Fairness and Environmental Sustainability in Natural Language Processingcited by 5
2023FairPy: A Toolkit for Evaluation of Prediction Biases and their Mitigation in Large Language Modelscited by 5
2024The Lou Dataset -- Exploring the Impact of Gender-Fair Language in German Text Classificationcited by 5
2022Fairness for Text Classification Tasks with Identity Information Data Augmentation Methodscited by 4
2023A Group Fairness Lens for Large Language Modelscited by 4
2024News Without Borders: Domain Adaptation of Multilingual Sentence Embeddings for Cross-lingual News Recommendationcited by 4
2022Counterfactual Multi-Token Fairness in Text Classificationcited by 3
2024The Life Cycle of Large Language Models: A Review of Biases in Educationcited by 3
2022Does Debiasing Inevitably Degrade the Model Performancecited by 2
2024UPCS: Unbiased Persona Construction for Dialogue Generationcited by 1
2024SoK: Towards Security and Safety of Edge AIcited by 1
2025Robust Bias Detection in MLMs and its Application to Human Trait Ratingscited by 1
2021Multi-Objective Few-shot Learning for Fair Classificationcited by 0
2023A General-Purpose Multilingual Document Encodercited by 0

50 most cited of 57 All 57 (JSON)