# How Does Counterfactually Augmented Data Impact Models for Social Computing Constructs?

arXiv: https://arxiv.org/abs/2109.07022 · HTML: https://mlsanity.com/paper/2109.07022/

## Key facts

- Cited by 19 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 14 Sep 2021.

## Built on by (most cited first, 19 of 19)

- [State-of-the-art generalisation research in NLP: A taxonomy and review](https://mlsanity.com/paper/2210.03050/) (2022)
- [A Survey on Gender Bias in Natural Language Processing](https://mlsanity.com/paper/2112.14168/) (2021)
- [A Survey on Multilingual Large Language Models: Corpora, Alignment, and Bias](https://mlsanity.com/paper/2404.00929/) (2024)
- [Perturbation Augmentation for Fairer NLP](https://mlsanity.com/paper/2205.12586/) (2022)
- [DoCoGen: Domain Counterfactual Generation for Low Resource Domain Adaptation](https://mlsanity.com/paper/2202.12350/) (2022)
- [LLMs for Generating and Evaluating Counterfactuals: A Comprehensive Study](https://mlsanity.com/paper/2405.00722/) (2024)
- [Choose Your Lenses: Flaws in Gender Bias Evaluation](https://mlsanity.com/paper/2210.11471/) (2022)
- [People Make Better Edits: Measuring the Efficacy of LLM-Generated Counterfactually Augmented Data for Harmful Language Detection](https://mlsanity.com/paper/2311.01270/) (2023)
- [NeuroCounterfactuals: Beyond Minimal-Edit Counterfactuals for Richer Data Augmentation](https://mlsanity.com/paper/2210.12365/) (2022)
- [Looking for a Handsome Carpenter! Debiasing GPT-3 Job Advertisements](https://mlsanity.com/paper/2205.11374/) (2022)
- [A Survey on Natural Language Counterfactual Generation](https://mlsanity.com/paper/2407.03993/) (2024)
- [CATfOOD: Counterfactual Augmented Training for Improving Out-of-Domain Performance and Calibration](https://mlsanity.com/paper/2309.07822/) (2023)
- [Machine Reading, Fast and Slow: When Do Models "Understand" Language?](https://mlsanity.com/paper/2209.07430/) (2022)
- [Counterfactually Augmented Data and Unintended Bias: The Case of Sexism and Hate Speech Detection](https://mlsanity.com/paper/2205.04238/) (2022)
- [Revisiting Contextual Toxicity Detection in Conversations](https://mlsanity.com/paper/2111.12447/) (2021)
- [The State of Profanity Obfuscation in Natural Language Processing](https://mlsanity.com/paper/2210.07595/) (2022)
- [StyLEx: Explaining Style Using Human Lexical Annotations](https://mlsanity.com/paper/2210.07469/) (2022)
- [Effects of Human Adversarial and Affable Samples on BERT Generalization](https://mlsanity.com/paper/2310.08008/) (2023)
- [Constraint-Aware Counterfactual Editing for Aspect-Based Sentiment Analysis](https://mlsanity.com/paper/2607.13977/) (2026)

## Abstract (arXiv)

As NLP models are increasingly deployed in socially situated settings such as online abusive content detection, it is crucial to ensure that these models are robust. One way of improving model robustness is to generate counterfactually augmented data (CAD) for training models that can better learn to distinguish between core features and data artifacts. While models trained on this type of data have shown promising out-of-domain generalizability, it is still unclear what the sources of such improvements are. We investigate the benefits of CAD for social NLP models by focusing on three social computing constructs -- sentiment, sexism, and hate speech. Assessing the performance of models trained with and without CAD across different types of datasets, we find that while models trained on CAD show lower in-domain performance, they generalize better out-of-domain. We unpack this apparent discrepancy using machine explanations and find that CAD reduces model reliance on spurious features. Leveraging a novel typology of CAD to analyze their relationship with model performance, we find that CAD which acts on the construct directly or a diverse set of CAD leads to higher performance.