mlsanityData to 14 Oct 2026

Choose Your Lenses: Flaws in Gender Bias Evaluation

arXiv 2210.11471 ↗First posted 20 Oct 2022cs.CLcited by 29

Not on the map. Why?

Abstract (arXiv)

Considerable efforts to measure and mitigate gender bias in recent years have led to the introduction of an abundance of tasks, datasets, and metrics used in this vein. In this position paper, we assess the current paradigm of gender bias evaluation and identify several flaws in it. First, we highlight the importance of extrinsic bias metrics that measure how a model's performance on some task is affected by gender, as opposed to intrinsic evaluations of model representations, which are less strongly connected to specific harms to people interacting with systems. We find that only a few extrinsic metrics are measured in most studies, although more can be measured. Second, we find that datasets and metrics are often coupled, and discuss how their coupling hinders the ability to obtain reliable conclusions, and how one may decouple them. We then investigate how the choice of the dataset and its composition, as well as the choice of the metric, affect bias measurement, finding significant variations across each of them. Finally, we propose several guidelines for more reliable gender bias evaluation.

Key facts. Cited by 29 papers in the map (data through 14 Oct 2026). Builds on 53 of its 81 references that are in the map. First posted on arXiv 20 Oct 2022.

Builds on

53 of 81 references.

References · 53

  1. 2019RoBERTa: A Robustly Optimized BERT Pretraining Approachcited by 13,372
  2. 2021On the Opportunities and Risks of Foundation Modelscited by 4,033
  3. 2016Man is to Computer Programmer as Woman is to Homemaker? Debiasing Word Embeddingscited by 1,312
  4. 2020Language (Technology) is Power: A Critical Survey of "Bias" in NLPcited by 956
  5. 2020StereoSet: Measuring stereotypical bias in pretrained language modelscited by 924
  6. 2016Semantics derived automatically from language corpora contain human-like biasescited by 901
  7. 2020CrowS-Pairs: A Challenge Dataset for Measuring Social Biases in Masked Language Modelscited by 710
  8. 2018Mitigating Unwanted Biases with Adversarial Learningcited by 663
  9. 2018Gender Bias in Coreference Resolution: Evaluation and Debiasing Methodscited by 659
  10. 2019Designing and Interpreting Probes with Control Taskscited by 640
  11. 2018Gender Bias in Coreference Resolutioncited by 442
  12. 2017Men Also Like Shopping: Reducing Gender Bias Amplification using Corpus-level Constraintscited by 420
38 more
  1. 2021BOLD: Dataset and Metrics for Measuring Biases in Open-Ended Language Generationcited by 412
  2. 2019On Measuring Social Biases in Sentence Encoderscited by 398
  3. 2019Bias in Bios: A Case Study of Semantic Representation Bias in a High-Stakes Settingcited by 349
  4. 2019Mitigating Gender Bias in Natural Language Processing: Literature Reviewcited by 287
  5. 2019Measuring Bias in Contextualized Word Representationscited by 276
  6. 2019Lipstick on a Pig: Debiasing Methods Cover up Systematic Gender Biases in Word Embeddings But do not Remove Themcited by 268
  7. 2018Examining Gender and Race Bias in Two Hundred Sentiment Analysis Systemscited by 234
  8. 2020Information-Theoretic Probing with Minimum Description Lengthcited by 232
  9. 2020Measuring and Reducing Gendered Correlations in Pre-trained Modelscited by 224
  10. 2019Identifying and Reducing Gender Bias in Word-Level Language Modelscited by 214
  11. 2019Gender Bias in Contextualized Word Embeddingscited by 203
  12. 2019Counterfactual Data Augmentation for Mitigating Gender Stereotypes in Languages with Rich Morphologycited by 200
  13. 2018Adversarial Removal of Demographic Attributes from Text Datacited by 170
  14. 2020Detecting Emergent Intersectional Biases: Contextualized Word Embeddings Contain a Distribution of Human-like Biasescited by 167
  15. 2018Reducing Gender Bias in Abusive Language Detectioncited by 152
  16. 2019Queens are Powerful too: Mitigating Gender Bias in Dialogue Generationcited by 152
  17. 2020Intrinsic Bias Metrics Do Not Correlate with Application Biascited by 148
  18. 2021Harms of Gender Exclusivity and Challenges in Non-Binary Representation in Language Technologiescited by 128
  19. 2019On Measuring and Mitigating Biased Inferences of Word Embeddingscited by 112
  20. 2021Debiasing Pre-trained Contextualised Embeddingscited by 107
  21. 2019It's All in the Name: Mitigating Gender Bias with Name-Based Counterfactual Data Substitutioncited by 103
  22. 2021A Survey on Gender Bias in Natural Language Processingcited by 89
  23. 2020Reducing Gender Bias in Neural Machine Translation as a Domain Adaptation Problemcited by 86
  24. 2020Unmasking Contextual Stereotypes: Measuring and Mitigating BERT's Gender Biascited by 82
  25. 2018Towards Robust and Privacy-preserving Text Representationscited by 75
  26. 2022On the Intrinsic and Extrinsic Fairness Evaluation Metrics for Contextualized Language Representationscited by 75
  27. 2019Gender-preserving Debiasing for Pre-trained Word Embeddingscited by 61
  28. 2019Understanding Undesirable Word Embedding Associationscited by 59
  29. 2019Reducing Gender Bias in Word-Level Language Models with a Gender-Equalizing Loss Functioncited by 57
  30. 2021Stereotype and Skew: Quantifying Gender Bias in Pre-trained and Fine-tuned Language Modelscited by 57
  31. 2020Mitigating Gender Bias in Machine Translation with Target Gender Annotationscited by 42
  32. 2020Neural Machine Translation Doesn't Translate Gender Coreference Right Unless You Make Itcited by 40
  33. 2021Dictionary-based Debiasing of Pre-trained Word Embeddingscited by 24
  34. 2020Nurse is Closer to Woman than Surgeon? Mitigating Gender-Biased Proximities in Word Embeddingscited by 20
  35. 2021Measuring Fairness with Biased Rulers: A Survey on Quantifying Biases in Pretrained Language Modelscited by 20
  36. 2019Debiasing Embeddings for Reduced Gender Bias in Text Classificationcited by 19
  37. 2021How Does Counterfactually Augmented Data Impact Models for Social Computing Constructs?cited by 19
  38. 2019Conceptor Debiasing of Word Representations Evaluated on WEATcited by 18

50 most cited of 53 All 53 (JSON)

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2022: 2; 2023: 10; 2024: 11; 2025: 3; 2026: 3.

29 papers.

Sort:

By area:

  1. 2023Bias and Fairness in Large Language Models: A Surveycited by 639
  2. 2024Monitoring Latent World States in Language Models with Propositional Probescited by 43
  3. 2023Shielded Representations: Protecting Sensitive Attributes Through Iterative Gradient-Based Projectioncited by 25
  4. 2024Investigating Implicit Bias in Large Language Models: A Large-Scale Study of Over 50 LLMscited by 25
  5. 2024Can LLMs Recognize Toxicity? A Structured Investigation Framework and Toxicity Metriccited by 23
  6. 2022BLIND: Bias Removal With No Demographicscited by 22
  7. 2023What about em? How Commercial Machine Translation Fails to Handle (Neo-)Pronounscited by 19
  8. 2024Akal Badi ya Bias: An Exploratory Study of Gender Bias in Hindi Language Technologycited by 17
  9. 2023Fairness in Language Models Beyond English: Gaps and Challengescited by 16
  10. 2024From 'Showgirls' to 'Performers': Fine-tuning with Gender-inclusive Language for Bias Reduction in LLMscited by 15
  11. 2024What the Harm? Quantifying the Tangible Impact of Gender Bias in Machine Translation with a Human-centered Studycited by 15
  12. 2023Women Are Beautiful, Men Are Leaders: Gender Stereotypes in Machine Translation and Language Modelingcited by 12
17 more
  1. 2023Target-Agnostic Gender-Aware Contrastive Learning for Mitigating Bias in Multilingual Machine Translationcited by 10
  2. 2023Comparing Intrinsic Gender Bias Evaluation Measures without using Human Annotated Examplescited by 8
  3. 2023In-Depth Look at Word Filling Societal Bias Measurescited by 8
  4. 2024How far can bias go? Tracing bias from pretraining data to alignmentcited by 5
  5. 2026Investigating Social Bias Changes in Quantized Language Modelscited by 5
  6. 2022Undesirable Biases in NLP: Addressing Challenges of Measurementcited by 4
  7. 2023A Group Fairness Lens for Large Language Modelscited by 4
  8. 2024RuBia: A Russian Language Bias Detection Datasetcited by 4
  9. 2024Addressing Both Statistical and Causal Gender Fairness in NLP Modelscited by 4
  10. 2024Leveraging Prototypical Representations for Mitigating Social Bias without Demographic Informationcited by 3
  11. 2025Is Your Model Fairly Certain? Uncertainty-Aware Fairness Evaluation for LLMscited by 2
  12. 2023Detecting and Mitigating Indirect Stereotypes in Word Embeddingscited by 0
  13. 2024MABR: Multilayer Adversarial Bias Removal Without Prior Bias Knowledgecited by 0
  14. 2025Investigating the Capabilities and Limitations of Machine Learning for Identifying Bias in English Language Data with Information and Heritage Professionalscited by 0
  15. 2025GPT and Prejudice: A Sparse Approach to Understanding Learned Representations in Large Language Modelscited by 0
  16. 2026Cross-Lingual Probing and Community-Grounded Analysis of Gender Bias in Low-Resource Bengalicited by 0
  17. 2026GKnow: Measuring the Entanglement of Gender Bias and Factual Gendercited by 0