# Mitigating Biases in Toxic Language Detection through Invariant Rationalization

arXiv: https://arxiv.org/abs/2106.07240 · HTML: https://mlsanity.com/paper/2106.07240/

## Key facts

- Cited by 7 papers in the map (data through 14 Oct 2026).
- Builds on 15 of its 24 references that are in the map.
- First posted on arXiv 14 Jun 2021.

## Builds on (most cited first, 15 of 15)

- [RoBERTa: A Robustly Optimized BERT Pretraining Approach](https://mlsanity.com/paper/1907.11692/) (2019)
- [Automated Hate Speech Detection and the Problem of Offensive Language](https://mlsanity.com/paper/1703.04009/) (2017)
- [Don't Take the Easy Way Out: Ensemble Based Methods for Avoiding Known Dataset Biases](https://mlsanity.com/paper/1909.03683/) (2019)
- [Large Scale Crowdsourcing and Characterization of Twitter Abusive Behavior](https://mlsanity.com/paper/1802.00393/) (2018)
- [Demographic Dialectal Variation in Social Media: A Case Study of African-American English](https://mlsanity.com/paper/1608.08868/) (2016)
- [Adversarial Filters of Dataset Biases](https://mlsanity.com/paper/2002.04108/) (2020)
- [Reducing Gender Bias in Abusive Language Detection](https://mlsanity.com/paper/1808.07231/) (2018)
- [Invariant Rationalization](https://mlsanity.com/paper/2003.09772/) (2020)
- [Unlearn Dataset Bias in Natural Language Inference by Fitting the Residual](https://mlsanity.com/paper/1908.10763/) (2019)
- [Challenges in Automated Debiasing for Toxic Language Detection](https://mlsanity.com/paper/2102.00086/) (2021)
- [Hate Speech Detection and Racial Bias Mitigation in Social Media based on BERT model](https://mlsanity.com/paper/2008.06460/) (2020)
- [Mind the Trade-off: Debiasing NLU Models without Degrading the In-distribution Performance](https://mlsanity.com/paper/2005.00315/) (2020)
- [Demoting Racial Bias in Hate Speech Detection](https://mlsanity.com/paper/2005.12246/) (2020)
- [Stereotypical Bias Removal for Hate Speech Detection Task using Knowledge-based Generalizations](https://mlsanity.com/paper/2001.05495/) (2020)
- [Detecting Hate Speech and Offensive Language on Twitter using Machine Learning: An N-gram and TFIDF based Approach](https://mlsanity.com/paper/1809.08651/) (2018)

## Built on by (most cited first, 7 of 7)

- [Are Shortest Rationales the Best Explanations for Human Understanding?](https://mlsanity.com/paper/2203.08788/) (2022)
- [The State of Profanity Obfuscation in Natural Language Processing](https://mlsanity.com/paper/2210.07595/) (2022)
- [VertAttack: Taking advantage of Text Classifiers' horizontal vision](https://mlsanity.com/paper/2404.08538/) (2024)
- [Investigating Bias In Automatic Toxic Comment Detection: An Empirical Study](https://mlsanity.com/paper/2108.06487/) (2021)
- [Dominant Set-based Active Learning for Text Classification and its Application to Online Social Media](https://mlsanity.com/paper/2202.00540/) (2022)
- [Suum Cuique: Studying Bias in Taboo Detection with a Community Perspective](https://mlsanity.com/paper/2203.11401/) (2022)
- [Bangla BERT for Hyperpartisan News Detection: A Semi-Supervised and Explainable AI Approach](https://mlsanity.com/paper/2507.21242/) (2025)

## Abstract (arXiv)

Automatic detection of toxic language plays an essential role in protecting social media users, especially minority groups, from verbal abuse. However, biases toward some attributes, including gender, race, and dialect, exist in most training datasets for toxicity detection. The biases make the learned models unfair and can even exacerbate the marginalization of people. Considering that current debiasing methods for general natural language understanding tasks cannot effectively mitigate the biases in the toxicity detectors, we propose to use invariant rationalization (InvRat), a game-theoretic framework consisting of a rationale generator and a predictor, to rule out the spurious correlation of certain syntactic patterns (e.g., identity mentions, dialect) to toxicity labels. We empirically show that our method yields lower false positive rate in both lexical and dialectal attributes than previous debiasing methods.