mlsanityData to 14 Oct 2026

Debiasing isn't enough! -- On the Effectiveness of Debiasing MLMs and their Social Biases in Downstream Tasks

arXiv 2210.02938 ↗First posted 6 Oct 2022cs.CLcited by 48

Not on the map. Why?

Abstract (arXiv)

We study the relationship between task-agnostic intrinsic and task-specific extrinsic social bias evaluation measures for Masked Language Models (MLMs), and find that there exists only a weak correlation between these two types of evaluation measures. Moreover, we find that MLMs debiased using different methods still re-learn social biases during fine-tuning on downstream tasks. We identify the social biases in both training instances as well as their assigned labels as reasons for the discrepancy between intrinsic and extrinsic bias evaluation measurements. Overall, our findings highlight the limitations of existing MLM bias evaluation measures and raise concerns on the deployment of MLMs in downstream applications using those measures.

Key facts. Cited by 48 papers in the map (data through 14 Oct 2026). Builds on 43 of its 58 references that are in the map. First posted on arXiv 6 Oct 2022.

Builds on

43 of 58 references.

References · 43

  1. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,399
  2. 2019RoBERTa: A Robustly Optimized BERT Pretraining Approachcited by 13,372
  3. 2019XLNet: Generalized Autoregressive Pretraining for Language Understandingcited by 2,811
  4. 2019ALBERT: A Lite BERT for Self-supervised Learning of Language Representationscited by 2,790
  5. 2017A Broad-Coverage Challenge Corpus for Sentence Understanding through Inferencecited by 2,682
  6. 2018Deep contextualized word representationscited by 2,290
  7. 2016Man is to Computer Programmer as Woman is to Homemaker? Debiasing Word Embeddingscited by 1,312
  8. 2017SemEval-2017 Task 1: Semantic Textual Similarity - Multilingual and Cross-lingual Focused Evaluationcited by 1,001
  9. 2020StereoSet: Measuring stereotypical bias in pretrained language modelscited by 924
  10. 2016Semantics derived automatically from language corpora contain human-like biasescited by 901
  11. 2020CrowS-Pairs: A Challenge Dataset for Measuring Social Biases in Masked Language Modelscited by 710
  12. 2020Null It Out: Guarding Protected Attributes by Iterative Nullspace Projectioncited by 455
31 more
  1. 2018Gender Bias in Coreference Resolutioncited by 442
  2. 2021BOLD: Dataset and Metrics for Measuring Biases in Open-Ended Language Generationcited by 412
  3. 2020HateXplain: A Benchmark Dataset for Explainable Hate Speech Detectioncited by 407
  4. 2019On Measuring Social Biases in Sentence Encoderscited by 398
  5. 2019Bias in Bios: A Case Study of Semantic Representation Bias in a High-Stakes Settingcited by 349
  6. 2019Measuring Bias in Contextualized Word Representationscited by 276
  7. 2019Lipstick on a Pig: Debiasing Methods Cover up Systematic Gender Biases in Word Embeddings But do not Remove Themcited by 268
  8. 2018Examining Gender and Race Bias in Two Hundred Sentiment Analysis Systemscited by 234
  9. 2020Measuring and Reducing Gendered Correlations in Pre-trained Modelscited by 224
  10. 2019Counterfactual Data Augmentation for Mitigating Gender Stereotypes in Languages with Rich Morphologycited by 200
  11. 2019BERT has a Mouth, and It Must Speak: BERT as a Markov Random Field Language Modelcited by 185
  12. 2020Towards Debiasing Sentence Representationscited by 171
  13. 2018Learning Gender-Neutral Word Embeddingscited by 169
  14. 2020Detecting Emergent Intersectional Biases: Contextualized Word Embeddings Contain a Distribution of Human-like Biasescited by 167
  15. 2020Intrinsic Bias Metrics Do Not Correlate with Application Biascited by 148
  16. 2021Harms of Gender Exclusivity and Challenges in Non-Binary Representation in Language Technologiescited by 128
  17. 2019On Measuring and Mitigating Biased Inferences of Word Embeddingscited by 112
  18. 2021Debiasing Pre-trained Contextualised Embeddingscited by 107
  19. 2019It's All in the Name: Mitigating Gender Bias with Name-Based Counterfactual Data Substitutioncited by 103
  20. 2021Sustainable Modular Debiasing of Language Modelscited by 100
  21. 2020Unmasking Contextual Stereotypes: Measuring and Mitigating BERT's Gender Biascited by 82
  22. 2022On the Intrinsic and Extrinsic Fairness Evaluation Metrics for Contextualized Language Representationscited by 75
  23. 2019Gender-preserving Debiasing for Pre-trained Word Embeddingscited by 61
  24. 2021Unmasking the Mask -- Evaluating Social Biases in Masked Language Modelscited by 60
  25. 2022Gender Bias in Masked Language Models for Multiple Languagescited by 60
  26. 2020Gender Bias in Multilingual Embeddings and Cross-Lingual Transfercited by 55
  27. 2021Dictionary-based Debiasing of Pre-trained Word Embeddingscited by 24
  28. 2021Measuring Fairness with Biased Rulers: A Survey on Quantifying Biases in Pretrained Language Modelscited by 20
  29. 2022Sense Embeddings are also Biased--Evaluating Social Biases in Static and Contextualised Sense Embeddingscited by 18
  30. 2020Autoencoding Improves Pre-trained Word Embeddingscited by 11
  31. 2022Gender Bias in Meta-Embeddingscited by 6

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2022: 2; 2023: 19; 2024: 15; 2025: 8; 2026: 4.

48 papers.

Sort:

By area:

  1. 2023Bias and Fairness in Large Language Models: A Surveycited by 639
  2. 2023Word Embeddings Are Steers for Language Modelscited by 70
  3. 2024Bias in Language Models: Beyond Trick Tests and Toward RUTEd Evaluationcited by 41
  4. 2024Social Bias Evaluation for Large Language Models Requires Prompt Variationscited by 40
  5. 2022MABEL: Attenuating Gender Bias using Textual Entailment Datacited by 31
  6. 2023Dissociating language and thought in large language modelscited by 29
  7. 2023On the Independence of Association Bias and Empirical Fairness in Language Modelscited by 29
  8. 2024Twists, Humps, and Pebbles: Multilingual Speech Recognition Models Exhibit Gender Performance Gapscited by 25
  9. 2024BiasAlert: A Plug-and-play Tool for Social Bias Detection in LLMscited by 21
  10. 2023Understanding the Effect of Model Compression on Social Bias in Large Language Modelscited by 19
  11. 2023Evaluating Gender Bias of Pre-trained Language Models in Natural Language Inference by Considering All Labelscited by 16
  12. 2024On the Intrinsic Self-Correction Capability of LLMs: Uncertainty and Latent Conceptcited by 16
36 more
  1. 2023Co2PT: Mitigating Bias in Pre-trained Language Models through Counterfactual Contrastive Prompt Tuningcited by 15
  2. 2025Safety Alignment via Constrained Knowledge Unlearningcited by 14
  3. 2023In-Contextual Gender Bias Suppression for Large Language Modelscited by 12
  4. 2023Comparing Intrinsic Gender Bias Evaluation Measures without using Human Annotated Examplescited by 8
  5. 2025Evaluating the Effect of Retrieval Augmentation on Social Biasescited by 7
  6. 2023Evaluating Bias and Fairness in Gender-Neutral Pretrained Vision-and-Language Modelscited by 6
  7. 2023On the Origins of Bias in NLP through the Lens of the Jim Codecited by 5
  8. 2023Solving NLP Problems through Human-System Collaboration: A Discussion-based Approachcited by 5
  9. 2024The Gaps between Pre-train and Downstream Settings in Bias Evaluation and Debiasingcited by 5
  10. 2024Smaller Large Language Models Can Do Moral Self-Correctioncited by 5
  11. 2026Investigating Social Bias Changes in Quantized Language Modelscited by 5
  12. 2023A Predictive Factor Analysis of Social Biases and Task-Performance in Pretrained Masked Language Modelscited by 4
  13. 2023Fair Text Classification with Wasserstein Independencecited by 4
  14. 2024Eagle: Ethical Dataset Given from Real Interactionscited by 4
  15. 2024Why Don't Prompt-Based Fairness Metrics Correlate?cited by 4
  16. 2024A Study of Nationality Bias in Names and Perplexity using Off-the-Shelf Affect-related Tweet Classifierscited by 4
  17. 2025Exploring Gender Bias in Large Language Models: An In-depth Dive into the German Languagecited by 3
  18. 2025Bias after Prompting: Persistent Discrimination in Large Language Modelscited by 3
  19. 2025Online Learning Defense against Iterative Jailbreak Attacks via Prompt Optimizationcited by 3
  20. 2023Thesis Distillation: Investigating The Impact of Bias in NLP Models on Hate Speech Detectioncited by 2
  21. 2023General Phrase Debiaser: Debiasing Masked Language Models at a Multi-Token Levelcited by 2
  22. 2024Towards Understanding Task-agnostic Debiasing Through the Lenses of Intrinsic Bias and Forgetfulnesscited by 2
  23. 2024Anna Karenina Strikes Again: Pre-Trained LLM Embeddings May Favor High-Performing Learnerscited by 2
  24. 2024Evaluating Gender Bias Transfer between Pre-trained and Prompt-Adapted Language Modelscited by 2
  25. 2022Gender Biases Unexpectedly Fluctuate in the Pre-training Stage of Masked Language Modelscited by 1
  26. 2023On Bias and Fairness in NLP: Investigating the Impact of Bias and Debiasing in Language Models on the Fairness of Toxicity Detectioncited by 1
  27. 2023Selecting Shots for Demographic Fairness in Few-Shot Learning with Large Language Modelscited by 1
  28. 2023Does Pre-trained Language Model Actually Infer Unseen Links in Knowledge Graph Completion?cited by 1
  29. 2024A Note on Bias to Completecited by 1
  30. 2025Fairness Dynamics During Trainingcited by 1
  31. 2025EsBBQ and CaBBQ: The Spanish and Catalan Bias Benchmarks for Question Answeringcited by 1
  32. 2024Semantic Properties of cosine based bias scores for word embeddingscited by 0
  33. 2025Textual Entailment is not a Better Bias Metric than Token Probabilitycited by 0
  34. 2026JPU: Bridging Jailbreak Defense and Unlearning via On-Policy Path Rectificationcited by 0
  35. 2026Cross-Lingual Probing and Community-Grounded Analysis of Gender Bias in Low-Resource Bengalicited by 0
  36. 2026Context-Aware Counterfactual Data Augmentation for Gender Bias Mitigation in Language Modelscited by 0