# Reducing Gender Bias in Abusive Language Detection

arXiv: https://arxiv.org/abs/1808.07231 · HTML: https://mlsanity.com/paper/1808.07231/

## Key facts

- Cited by 152 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 22 Aug 2018.

## Built on by (most cited first, 30 of 152)

- [XSTest: A Test Suite for Identifying Exaggerated Safety Behaviours in Large Language Models](https://mlsanity.com/paper/2308.01263/) (2023)
- [Text and Code Embeddings by Contrastive Pre-Training](https://mlsanity.com/paper/2201.10005/) (2022)
- [From Pretraining Data to Language Models to Downstream Tasks: Tracking the Trails of Political Biases Leading to Unfair NLP Models](https://mlsanity.com/paper/2305.08283/) (2023)
- [Societal Biases in Language Generation: Progress and Challenges](https://mlsanity.com/paper/2105.04054/) (2021)
- [Gender bias and stereotypes in Large Language Models](https://mlsanity.com/paper/2308.14921/) (2023)
- [Challenges in Detoxifying Language Models](https://mlsanity.com/paper/2109.07445/) (2021)
- [State-of-the-art generalisation research in NLP: A taxonomy and review](https://mlsanity.com/paper/2210.03050/) (2022)
- [Challenges in Automated Debiasing for Toxic Language Detection](https://mlsanity.com/paper/2102.00086/) (2021)
- [A Survey on Fairness in Large Language Models](https://mlsanity.com/paper/2308.10149/) (2023)
- [Trustworthy AI: A Computational Perspective](https://mlsanity.com/paper/2107.06641/) (2021)
- [Large Language Models are Geographically Biased](https://mlsanity.com/paper/2402.02680/) (2024)
- [A Survey on Gender Bias in Natural Language Processing](https://mlsanity.com/paper/2112.14168/) (2021)
- [Mitigating Language-Dependent Ethnic Bias in BERT](https://mlsanity.com/paper/2109.05704/) (2021)
- [On Measures of Biases and Harms in NLP](https://mlsanity.com/paper/2108.03362/) (2021)
- [Bias in Large Language Models: Origin, Evaluation, and Mitigation](https://mlsanity.com/paper/2411.10915/) (2024)
- [Mitigating Political Bias in Language Models Through Reinforced Calibration](https://mlsanity.com/paper/2104.14795/) (2021)
- [Human-AI Collaboration via Conditional Delegation: A Case Study of Content Moderation](https://mlsanity.com/paper/2204.11788/) (2022)
- [Towards generalisable hate speech detection: a review on obstacles and solutions](https://mlsanity.com/paper/2102.08886/) (2021)
- [Theories of "Gender" in NLP Bias Research](https://mlsanity.com/paper/2205.02526/) (2022)
- [Handling Bias in Toxic Speech Detection: A Survey](https://mlsanity.com/paper/2202.00126/) (2022)
- [Entropy-based Attention Regularization Frees Unintended Bias Mitigation from Lists](https://mlsanity.com/paper/2203.09192/) (2022)
- [Explanation-Based Human Debugging of NLP Models: A Survey](https://mlsanity.com/paper/2104.15135/) (2021)
- [ConvAbuse: Data, Analysis, and Benchmarks for Nuanced Abuse Detection in Conversational AI](https://mlsanity.com/paper/2109.09483/) (2021)
- [Evaluating GPT-3 Generated Explanations for Hateful Content Moderation](https://mlsanity.com/paper/2305.17680/) (2023)
- [MoCoDA: Model-based Counterfactual Data Augmentation](https://mlsanity.com/paper/2210.11287/) (2022)
- [Fairness for Unobserved Characteristics: Insights from Technological Impacts on Queer Communities](https://mlsanity.com/paper/2102.04257/) (2021)
- [Quantifying Social Biases Using Templates is Unreliable](https://mlsanity.com/paper/2210.04337/) (2022)
- [Representation Bias in Data: A Survey on Identification and Resolution Techniques](https://mlsanity.com/paper/2203.11852/) (2022)
- [Balancing out Bias: Achieving Fairness Through Balanced Training](https://mlsanity.com/paper/2109.08253/) (2021)
- [Modeling the Data-Generating Process is Necessary for Out-of-Distribution Generalization](https://mlsanity.com/paper/2206.07837/) (2022)

## Abstract (arXiv)

Abusive language detection models tend to have a problem of being biased toward identity words of a certain group of people because of imbalanced training datasets. For example, "You are a good woman" was considered "sexist" when trained on an existing dataset. Such model bias is an obstacle for models to be robust enough for practical use. In this work, we measure gender biases on models trained with different abusive language datasets, while analyzing the effect of different pre-trained word embeddings and model architectures. We also experiment with three bias mitigation methods: (1) debiased word embeddings, (2) gender swap data augmentation, and (3) fine-tuning with a larger corpus. These methods can effectively reduce gender bias by 90-98% and can be extended to correct model bias in other scenarios.