# Debiasing should be Good and Bad: Measuring the Consistency of Debiasing Techniques in Language Models

arXiv: https://arxiv.org/abs/2305.14307 · HTML: https://mlsanity.com/paper/2305.14307/

## Key facts

- Cited by 7 papers in the map (data through 14 Oct 2026).
- Builds on 28 of its 51 references that are in the map.
- First posted on arXiv 23 May 2023.

## Builds on (most cited first, 28 of 28)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [Language Models are Few-Shot Learners](https://mlsanity.com/paper/2005.14165/) (2020)
- [Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformer](https://mlsanity.com/paper/1910.10683/) (2019)
- [RoBERTa: A Robustly Optimized BERT Pretraining Approach](https://mlsanity.com/paper/1907.11692/) (2019)
- [PaLM: Scaling Language Modeling with Pathways](https://mlsanity.com/paper/2204.02311/) (2022)
- [Switch Transformers: Scaling to Trillion Parameter Models with Simple and Efficient Sparsity](https://mlsanity.com/paper/2101.03961/) (2021)
- [Man is to Computer Programmer as Woman is to Homemaker? Debiasing Word Embeddings](https://mlsanity.com/paper/1607.06520/) (2016)
- [RealToxicityPrompts: Evaluating Neural Toxic Degeneration in Language Models](https://mlsanity.com/paper/2009.11462/) (2020)
- [Beyond Accuracy: Behavioral Testing of NLP models with CheckList](https://mlsanity.com/paper/2005.04118/) (2020)
- [StereoSet: Measuring stereotypical bias in pretrained language models](https://mlsanity.com/paper/2004.09456/) (2020)
- [CrowS-Pairs: A Challenge Dataset for Measuring Social Biases in Masked Language Models](https://mlsanity.com/paper/2010.00133/) (2020)
- [The Woman Worked as a Babysitter: On Biases in Language Generation](https://mlsanity.com/paper/1909.01326/) (2019)
- [Self-Diagnosis and Self-Debiasing: A Proposal for Reducing Corpus-Based Bias in NLP](https://mlsanity.com/paper/2103.00453/) (2021)
- [Probing Image-Language Transformers for Verb Understanding](https://mlsanity.com/paper/2106.09141/) (2021)
- [Debiasing Pre-trained Contextualised Embeddings](https://mlsanity.com/paper/2101.09523/) (2021)
- [A Prompt Array Keeps the Bias Away: Debiasing Vision-Language Models with Adversarial Learning](https://mlsanity.com/paper/2203.11933/) (2022)
- [Sustainable Modular Debiasing of Language Models](https://mlsanity.com/paper/2109.03646/) (2021)
- [Logic-Guided Data Augmentation and Regularization for Consistent Question Answering](https://mlsanity.com/paper/2004.10157/) (2020)
- [Reducing Gender Bias in Word-Level Language Models with a Gender-Equalizing Loss Function](https://mlsanity.com/paper/1905.12801/) (2019)
- [Word Order Does Matter (And Shuffled Language Models Know It)](https://mlsanity.com/paper/2203.10995/) (2022)
- [Defining and Detecting Toxicity on Social Media: Context and Knowledge are Key](https://mlsanity.com/paper/2104.10788/) (2021)
- [Dictionary-based Debiasing of Pre-trained Word Embeddings](https://mlsanity.com/paper/2101.09525/) (2021)
- [How Reasonable are Common-Sense Reasoning Tasks: A Case-Study on the Winograd Schema Challenge and SWAG](https://mlsanity.com/paper/1811.01778/) (2018)
- [Probing for Bridging Inference in Transformer Language Models](https://mlsanity.com/paper/2104.09400/) (2021)
- [Probing for Multilingual Numerical Understanding in Transformer-Based Language Models](https://mlsanity.com/paper/2010.06666/) (2020)
- [Correlation-based Intrinsic Evaluation of Word Vector Representations](https://mlsanity.com/paper/1606.06710/) (2016)
- [Towards Ethical Machines Via Logic Programming](https://mlsanity.com/paper/1909.08255/) (2019)

## Built on by (most cited first, 7 of 7)

- [Leak, Cheat, Repeat: Data Contamination and Evaluation Malpractices in Closed-Source LLMs](https://mlsanity.com/paper/2402.03927/) (2024)
- [Can LLMs Recognize Toxicity? A Structured Investigation Framework and Toxicity Metric](https://mlsanity.com/paper/2402.06900/) (2024)
- ['Since Lawyers are Males..': Examining Implicit Gender Bias in Hindi Language Generation by LLMs](https://mlsanity.com/paper/2409.13484/) (2024)
- [Pitfalls of Conversational LLMs on News Debiasing](https://mlsanity.com/paper/2404.06488/) (2024)
- [Paying Alignment Tax with Contrastive Learning](https://mlsanity.com/paper/2505.19327/) (2025)
- [Bye Bye Perspective API: Lessons for Building and Governing Measurement Infrastructure](https://mlsanity.com/paper/2604.25580/) (2026)
- [Foundation Models at Work: Fine-Tuning for Fairness in Algorithmic Hiring](https://mlsanity.com/paper/2501.07324/) (2025)

## Abstract (arXiv)

Debiasing methods that seek to mitigate the tendency of Language Models (LMs) to occasionally output toxic or inappropriate text have recently gained traction. In this paper, we propose a standardized protocol which distinguishes methods that yield not only desirable results, but are also consistent with their mechanisms and specifications. For example, we ask, given a debiasing method that is developed to reduce toxicity in LMs, if the definition of toxicity used by the debiasing method is reversed, would the debiasing results also be reversed? We used such considerations to devise three criteria for our new protocol: Specification Polarity, Specification Importance, and Domain Transferability. As a case study, we apply our protocol to a popular debiasing method, Self-Debiasing, and compare it to one we propose, called Instructive Debiasing, and demonstrate that consistency is as important an aspect to debiasing viability as is simply a desirable result. We show that our protocol provides essential insights into the generalizability and interpretability of debiasing methods that may otherwise go overlooked.