# Counterfactually Augmented Data and Unintended Bias: The Case of Sexism and Hate Speech Detection

arXiv: https://arxiv.org/abs/2205.04238 · HTML: https://mlsanity.com/paper/2205.04238/

## Key facts

- Cited by 11 papers in the map (data through 14 Oct 2026).
- Builds on 18 of its 36 references that are in the map.
- First posted on arXiv 9 May 2022.

## Builds on (most cited first, 18 of 18)

- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [Beyond Accuracy: Behavioral Testing of NLP models with CheckList](https://mlsanity.com/paper/2005.04118/) (2020)
- [Language (Technology) is Power: A Critical Survey of "Bias" in NLP](https://mlsanity.com/paper/2005.14050/) (2020)
- [Learning the Difference that Makes a Difference with Counterfactually-Augmented Data](https://mlsanity.com/paper/1909.12434/) (2019)
- [Nuanced Metrics for Measuring Unintended Bias with Real Data for Text Classification](https://mlsanity.com/paper/1903.04561/) (2019)
- [Polyjuice: Generating Counterfactuals for Explaining, Evaluating, and Improving Models](https://mlsanity.com/paper/2101.00288/) (2021)
- [Learning from the Worst: Dynamically Generated Datasets to Improve Online Hate Detection](https://mlsanity.com/paper/2012.15761/) (2020)
- [HateCheck: Functional Tests for Hate Speech Detection Models](https://mlsanity.com/paper/2012.15606/) (2020)
- [Causal Inference in Natural Language Processing: Estimation, Prediction, Interpretation and Beyond](https://mlsanity.com/paper/2109.00725/) (2021)
- [Build it Break it Fix it for Dialogue Safety: Robustness from Adversarial Human Attack](https://mlsanity.com/paper/1908.06083/) (2019)
- [Contextualizing Hate Speech Classifiers with Post-hoc Explanation](https://mlsanity.com/paper/2005.02439/) (2020)
- [Removing Spurious Features can Hurt Accuracy and Affect Groups Disproportionately](https://mlsanity.com/paper/2012.04104/) (2020)
- [A Just and Comprehensive Strategy for Using NLP to Address Online Abuse](https://mlsanity.com/paper/1906.01738/) (2019)
- ["Call me sexist, but...": Revisiting Sexism Detection Using Psychological Scales and Adversarial Samples](https://mlsanity.com/paper/2004.12764/) (2020)
- [Causal Direction of Data Collection Matters: Implications of Causal and Anticausal Learning for NLP](https://mlsanity.com/paper/2110.03618/) (2021)
- [How Does Counterfactually Augmented Data Impact Models for Social Computing Constructs?](https://mlsanity.com/paper/2109.07022/) (2021)
- [Fact Checking with Insufficient Evidence](https://mlsanity.com/paper/2204.02007/) (2022)
- [Text as Causal Mediators: Research Design for Causal Estimates of Differential Treatment of Social Groups via Language Aspects](https://mlsanity.com/paper/2109.07542/) (2021)

## Built on by (most cited first, 11 of 11)

- [A Survey on Multilingual Large Language Models: Corpora, Alignment, and Bias](https://mlsanity.com/paper/2404.00929/) (2024)
- [Perturbation Augmentation for Fairer NLP](https://mlsanity.com/paper/2205.12586/) (2022)
- [Probing Classifiers are Unreliable for Concept Removal and Detection](https://mlsanity.com/paper/2207.04153/) (2022)
- [Hate Speech Detection with Generalizable Target-aware Fairness](https://mlsanity.com/paper/2406.00046/) (2024)
- [Hate Speech Detection via Dual Contrastive Learning](https://mlsanity.com/paper/2307.05578/) (2023)
- [Unlock the Potential of Counterfactually-Augmented Data in Out-Of-Distribution Generalization](https://mlsanity.com/paper/2310.06666/) (2023)
- [Divided by discipline? A systematic literature review on the quantification of online sexism and misogyny using a semi-automated approach](https://mlsanity.com/paper/2409.20204/) (2024)
- [Causal Effect Regularization: Automated Detection and Removal of Spurious Attributes](https://mlsanity.com/paper/2306.11072/) (2023)
- [On the Challenges of Building Datasets for Hate Speech Detection](https://mlsanity.com/paper/2309.02912/) (2023)
- [NLPGuard: A Framework for Mitigating the Use of Protected Attributes by NLP Classifiers](https://mlsanity.com/paper/2407.01697/) (2024)
- [Training-Time Explainability for Multilingual Hate Speech Detection: Aligning Model Reasoning with Human Rationales](https://mlsanity.com/paper/2608.26125/) (2026)

## Abstract (arXiv)

Counterfactually Augmented Data (CAD) aims to improve out-of-domain generalizability, an indicator of model robustness. The improvement is credited with promoting core features of the construct over spurious artifacts that happen to correlate with it. Yet, over-relying on core features may lead to unintended model bias. Especially, construct-driven CAD -- perturbations of core features -- may induce models to ignore the context in which core features are used. Here, we test models for sexism and hate speech detection on challenging data: non-hateful and non-sexist usage of identity and gendered terms. In these hard cases, models trained on CAD, especially construct-driven CAD, show higher false-positive rates than models trained on the original, unperturbed data. Using a diverse set of CAD -- construct-driven and construct-agnostic -- reduces such unintended bias.