mlsanityData to 7 Oct 2026

On the Origins of Bias in NLP through the Lens of the Jim Code

arXiv 2305.09281 ↗First posted 16 May 2023cs.CLcited by 5

Not on the map. Why?

Abstract (arXiv)

In this paper, we trace the biases in current natural language processing (NLP) models back to their origins in racism, sexism, and homophobia over the last 500 years. We review literature from critical race theory, gender studies, data ethics, and digital humanities studies, and summarize the origins of bias in NLP models from these social science perspective. We show how the causes of the biases in the NLP pipeline are rooted in social issues. Finally, we argue that the only way to fix the bias and unfairness in NLP is by addressing the social problems that caused them in the first place and by incorporating social sciences and social scientists in efforts to mitigate bias in NLP models. We provide actionable recommendations for the NLP research community to do so.

Key facts. Cited by 5 papers in the map (data through 7 Oct 2026). Builds on 23 of its 68 references that are in the map. First posted on arXiv 16 May 2023.

Builds on

23 of 68 references.

References · 23

  1. 2016Man is to Computer Programmer as Woman is to Homemaker? Debiasing Word Embeddingscited by 1,304
  2. 2020StereoSet: Measuring stereotypical bias in pretrained language modelscited by 911
  3. 2016Semantics derived automatically from language corpora contain human-like biasescited by 894
  4. 2017Counterfactual Fairnesscited by 802
  5. 2020CrowS-Pairs: A Challenge Dataset for Measuring Social Biases in Masked Language Modelscited by 703
  6. 2019On Measuring Social Biases in Sentence Encoderscited by 397
  7. 2020Fairness in Machine Learning: A Surveycited by 369
  8. 2021Self-Diagnosis and Self-Debiasing: A Proposal for Reducing Corpus-Based Bias in NLPcited by 336
  9. 2019Lipstick on a Pig: Debiasing Methods Cover up Systematic Gender Biases in Word Embeddings But do not Remove Themcited by 268
  10. 2017Word Embeddings Quantify 100 Years of Gender and Ethnic Stereotypescited by 251
  11. 2020Towards Debiasing Sentence Representationscited by 170
  12. 2020Intrinsic Bias Metrics Do Not Correlate with Application Biascited by 148
11 more
  1. 2019Predictive Biases in Natural Language Processing Models: A Conceptual Framework and Overviewcited by 137
  2. 201850 Years of Test (Un)fairness: Lessons for Machine Learningcited by 101
  3. 2021Quantifying Social Biases in NLP: A Generalization and Empirical Comparison of Extrinsic Fairness Metricscited by 83
  4. 2022On the Intrinsic and Extrinsic Fairness Evaluation Metrics for Contextualized Language Representationscited by 75
  5. 2019Perturbation Sensitivity Analysis to Detect Unintended Model Biasescited by 74
  6. 2022Perturbation Augmentation for Fairer NLPcited by 51
  7. 2022Debiasing isn't enough! -- On the Effectiveness of Debiasing MLMs and their Social Biases in Downstream Taskscited by 48
  8. 2021Fairness for Unobserved Characteristics: Insights from Technological Impacts on Queer Communitiescited by 40
  9. 2021Your fairness may vary: Pretrained language model fairness in toxic text classificationcited by 38
  10. 2022What Do NLP Researchers Believe? Results of the NLP Community Metasurveycited by 28
  11. 2021How Could Equality and Data Protection Law Shape AI Fairness for People with Disabilities?cited by 1

Built on by

Cited by year

Citing papers per year: 2023: 2; 2024: 3.

5 papers.

  1. 2023NBIAS: A Natural Language Processing Framework for Bias Identification in Textcited by 23
  2. 2024QueerBench: Quantifying Discrimination in Language Models Toward Queer Identitiescited by 5
  3. 2023Thesis Distillation: Investigating The Impact of Bias in NLP Models on Hate Speech Detectioncited by 2
  4. 2024Towards Understanding Task-agnostic Debiasing Through the Lenses of Intrinsic Bias and Forgetfulnesscited by 2
  5. 2024Computational Analysis of Character Development in Holocaust Testimoniescited by 1