# Demographics Should Not Be the Reason of Toxicity: Mitigating Discrimination in Text Classifications with Instance Weighting

arXiv: https://arxiv.org/abs/2004.14088 · HTML: https://mlsanity.com/paper/2004.14088/

## Key facts

- Cited by 41 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 29 Apr 2020.

## Built on by (most cited first, 30 of 41)

- [A Survey on Fairness in Large Language Models](https://mlsanity.com/paper/2308.10149/) (2023)
- [Trustworthy AI: A Computational Perspective](https://mlsanity.com/paper/2107.06641/) (2021)
- [A Survey of Race, Racism, and Anti-Racism in NLP](https://mlsanity.com/paper/2106.11410/) (2021)
- [On the Intrinsic and Extrinsic Fairness Evaluation Metrics for Contextualized Language Representations](https://mlsanity.com/paper/2203.13928/) (2022)
- [Theories of "Gender" in NLP Bias Research](https://mlsanity.com/paper/2205.02526/) (2022)
- [Handling Bias in Toxic Speech Detection: A Survey](https://mlsanity.com/paper/2202.00126/) (2022)
- [Entropy-based Attention Regularization Frees Unintended Bias Mitigation from Lists](https://mlsanity.com/paper/2203.09192/) (2022)
- [Fairness Reprogramming](https://mlsanity.com/paper/2209.10222/) (2022)
- [Sociodemographic Bias in Language Models: A Survey and Forward Path](https://mlsanity.com/paper/2306.08158/) (2023)
- [Deep Learning on a Healthy Data Diet: Finding Important Examples for Fairness](https://mlsanity.com/paper/2211.11109/) (2022)
- [A Causal View of Entity Bias in (Large) Language Models](https://mlsanity.com/paper/2305.14695/) (2023)
- [Does Robustness Improve Fairness? Approaching Fairness with Word Substitution Robustness Methods for Text Classification](https://mlsanity.com/paper/2106.10826/) (2021)
- [On Modality Bias Recognition and Reduction](https://mlsanity.com/paper/2202.12690/) (2022)
- [The Authors Matter: Understanding and Mitigating Implicit Bias in Deep Text Classification](https://mlsanity.com/paper/2105.02778/) (2021)
- [Enhancing Model Robustness and Fairness with Causality: A Regularization Approach](https://mlsanity.com/paper/2110.00911/) (2021)
- [Explainable Abuse Detection as Intent Classification and Slot Filling](https://mlsanity.com/paper/2210.02659/) (2022)
- [Should We Attend More or Less? Modulating Attention for Fairness](https://mlsanity.com/paper/2305.13088/) (2023)
- [An Invariant Learning Characterization of Controlled Text Generation](https://mlsanity.com/paper/2306.00198/) (2023)
- [Why only Micro-F1? Class Weighting of Measures for Relation Classification](https://mlsanity.com/paper/2205.09460/) (2022)
- [Target Span Detection for Implicit Harmful Content](https://mlsanity.com/paper/2403.19836/) (2024)
- [Hate Speech Detection with Generalizable Target-aware Fairness](https://mlsanity.com/paper/2406.00046/) (2024)
- [Detecting Inappropriate Messages on Sensitive Topics that Could Harm a Company's Reputation](https://mlsanity.com/paper/2103.05345/) (2021)
- [Wisdom from Diversity: Bias Mitigation Through Hybrid Human-LLM Crowds](https://mlsanity.com/paper/2505.12349/) (2025)
- [Easy Adaptation to Mitigate Gender Bias in Multilingual Text Classification](https://mlsanity.com/paper/2204.05459/) (2022)
- [Toward Understanding Bias Correlations for Mitigation in NLP](https://mlsanity.com/paper/2205.12391/) (2022)
- [Power of Explanations: Towards automatic debiasing in hate speech detection](https://mlsanity.com/paper/2209.09975/) (2022)
- [Bias in, Bias out: Annotation Bias in Multilingual Large Language Models](https://mlsanity.com/paper/2511.14662/) (2025)
- [On LLM Wizards: Identifying Large Language Models' Behaviors for Wizard of Oz Experiments](https://mlsanity.com/paper/2407.08067/) (2024)
- [Enriching Abusive Language Detection with Community Context](https://mlsanity.com/paper/2206.08445/) (2022)
- [Towards a Deep Multi-layered Dialectal Language Analysis: A Case Study of African-American English](https://mlsanity.com/paper/2206.08978/) (2022)

## Abstract (arXiv)

With the recent proliferation of the use of text classifications, researchers have found that there are certain unintended biases in text classification datasets. For example, texts containing some demographic identity-terms (e.g., "gay", "black") are more likely to be abusive in existing abusive language detection datasets. As a result, models trained with these datasets may consider sentences like "She makes me happy to be gay" as abusive simply because of the word "gay." In this paper, we formalize the unintended biases in text classification datasets as a kind of selection bias from the non-discrimination distribution to the discrimination distribution. Based on this formalization, we further propose a model-agnostic debiasing training framework by recovering the non-discrimination distribution using instance weighting, which does not require any extra resources or annotations apart from a pre-defined set of demographic identity-terms. Experiments demonstrate that our method can effectively alleviate the impacts of the unintended biases without significantly hurting models' generalization ability.