mlsanityData to 7 Oct 2026

A Survey of Race, Racism, and Anti-Racism in NLP

arXiv 2106.11410 ↗First posted 21 Jun 2021cs.CLcited by 74

Not on the map. Why?

Abstract (arXiv)

Despite inextricable ties between race and language, little work has considered race in NLP research and development. In this work, we survey 79 papers from the ACL anthology that mention race. These papers reveal various types of race-related bias in all stages of NLP model development, highlighting the need for proactive consideration of how NLP systems can uphold racial hierarchies. However, persistent gaps in research on race and NLP remain: race has been siloed as a niche topic and remains ignored in many NLP tasks; most work operationalizes race as a fixed single-dimensional variable with a ground-truth label, which risks reinforcing differences produced by historical racism; and the voices of historically marginalized people are nearly absent in NLP literature. By identifying where and how NLP literature has and has not considered race, especially in comparison to related fields, our work calls for inclusion and racial justice in NLP research practices.

Key facts. Cited by 74 papers in the map (data through 7 Oct 2026). Builds on 57 of its 150 references that are in the map. First posted on arXiv 21 Jun 2021.

Builds on

57 of 150 references.

References · 57

  1. 2016Man is to Computer Programmer as Woman is to Homemaker? Debiasing Word Embeddingscited by 1,304
  2. 2019Energy and Policy Considerations for Deep Learning in NLPcited by 1,242
  3. 2020Language (Technology) is Power: A Critical Survey of "Bias" in NLPcited by 951
  4. 2020StereoSet: Measuring stereotypical bias in pretrained language modelscited by 911
  5. 2016Semantics derived automatically from language corpora contain human-like biasescited by 894
  6. 2020The State and Fate of Linguistic Diversity and Inclusion in the NLP Worldcited by 789
  7. 2019Universal Adversarial Triggers for Attacking and Analyzing NLPcited by 709
  8. 2020CrowS-Pairs: A Challenge Dataset for Measuring Social Biases in Masked Language Modelscited by 703
  9. 2019The Woman Worked as a Babysitter: On Biases in Language Generationcited by 540
  10. 2017Men Also Like Shopping: Reducing Gender Bias Amplification using Corpus-level Constraintscited by 420
  11. 2019On Measuring Social Biases in Sentence Encoderscited by 397
  12. 2019Social Bias Frames: Reasoning about Social and Power Implications of Languagecited by 377
38 more
  1. 2019Bias in Bios: A Case Study of Semantic Representation Bias in a High-Stakes Settingcited by 346
  2. 2019Mitigating Gender Bias in Natural Language Processing: Literature Reviewcited by 287
  3. 2019Measuring Bias in Contextualized Word Representationscited by 275
  4. 2017Word Embeddings Quantify 100 Years of Gender and Ethnic Stereotypescited by 251
  5. 2018Examining Gender and Race Bias in Two Hundred Sentiment Analysis Systemscited by 234
  6. 2019Racial Bias in Hate Speech and Abusive Language Detection Datasetscited by 208
  7. 2018Hate Speech Dataset from a White Supremacy Forumcited by 196
  8. 2016Demographic Dialectal Variation in Social Media: A Case Study of African-American Englishcited by 194
  9. 2020Social Biases in NLP Models as Barriers for Persons with Disabilitiescited by 182
  10. 2018Adversarial Removal of Demographic Attributes from Text Datacited by 166
  11. 2019Black is to Criminal as Caucasian is to Police: Detecting and Removing Multiclass Bias in Word Embeddingscited by 146
  12. 2019Predictive Biases in Natural Language Processing Models: A Conceptual Framework and Overviewcited by 137
  13. 2019Assessing Social and Intersectional Biases in Contextualized Word Representationscited by 134
  14. 2019Towards a Critical Race Methodology in Algorithmic Fairnesscited by 124
  15. 2017Understanding Abuse: A Typology of Abusive Language Detection Subtaskscited by 115
  16. 2020Participation is not a Design Fix for Machine Learningcited by 110
  17. 2019Does Gender Matter? Towards Fairness in Dialogue Systemscited by 89
  18. 2020Contextualizing Hate Speech Classifiers with Post-hoc Explanationcited by 84
  19. 2019Diversity in Facescited by 77
  20. 2019Analyzing Polarization in Social Media: Method and Application to Tweets on 21 Mass Shootingscited by 67
  21. 2020It's Morphin' Time! Combating Linguistic Discrimination with Inflectional Perturbationscited by 64
  22. 2020Demoting Racial Bias in Hate Speech Detectioncited by 62
  23. 2020Investigating African-American Vernacular English in Transformer-Based Text Generationcited by 52
  24. 2019What's in a Name? Reducing Bias in Bios without Access to Protected Attributescited by 51
  25. 2019A Just and Comprehensive Strategy for Using NLP to Address Online Abusecited by 49
  26. 2020Multilingual Twitter Corpus and Baselines for Evaluating Demographic Bias in Hate Speech Recognitioncited by 43
  27. 2020Demographics Should Not Be the Reason of Toxicity: Mitigating Discrimination in Text Classifications with Instance Weightingcited by 41
  28. 2021Disembodied Machine Learning: On the Illusion of Objectivity in NLPcited by 37
  29. 2019Are We Consistently Biased? Multidimensional Analysis of Biases in Distributional Word Vectorscited by 31
  30. 2020A Framework for the Computational Linguistic Analysis of Dehumanizationcited by 31
  31. 2020Give Me Convenience and Give Her Death: Who Should Decide What Uses of NLP are Appropriate, and on What Basis?cited by 27
  32. 2020Weakly Supervised Learning of Nuanced Frames for Analyzing Polarization in News Mediacited by 27
  33. 2016Anchored Correlation Explanation: Topic Modeling with Minimal Domain Knowledgecited by 23
  34. 2021Machine Translation into Low-resource Language Varietiescited by 19
  35. 2019Conceptor Debiasing of Word Representations Evaluated on WEATcited by 18
  36. 2020Controlled Analyses of Social Biases in Wikipedia Bioscited by 13
  37. 2017Cross-linguistic differences and similarities in image descriptionscited by 12
  38. 2020Unequal Representations: Analyzing Intersectional Biases in Word Embeddings Using Representational Similarity Analysiscited by 12

50 most cited of 57 All 57 (JSON)

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2021: 2; 2022: 15; 2023: 18; 2024: 20; 2025: 13; 2026: 6.

74 papers.

Sort:

By area:

  1. 2023From Pretraining Data to Language Models to Downstream Tasks: Tracking the Trails of Political Biases Leading to Unfair NLP Modelscited by 257
  2. 2023Evaluating the Social Impact of Generative AI Systems in Systems and Societycited by 91
  3. 2022Language Generation Models Can Cause Harm: So What Can We Do About It? An Actionable Surveycited by 80
  4. 2023Eight Things to Know about Large Language Modelscited by 73
  5. 2022Perturbation Augmentation for Fairer NLPcited by 51
  6. 2022Fairness Reprogrammingcited by 33
  7. 2022Data Governance in the Age of Large-Scale Data-Driven Language Technologycited by 32
  8. 2022GeoMLAMA: Geo-Diverse Commonsense Probing on Multilingual Pre-Trained Language Modelscited by 30
  9. 2023The ROOTS Search Tool: Data Transparency for LLMscited by 30
  10. 2021Influence Tuning: Demoting Spurious Correlations via Instance Attribution and Instance-Driven Updatescited by 27
  11. 2024Do Generative AI Models Output Harm while Representing Non-Western Cultures: Evidence from A Community-Centered Approachcited by 25
  12. 2021Hi, my name is Martha: Using names to measure and mitigate bias in generative dialogue modelscited by 23
38 more
  1. 2022How Hate Speech Varies by Target Identity: A Computational Analysiscited by 23
  2. 2024Stop! In the Name of Flaws: Disentangling Personal Names and Sociodemographic Attributes in NLPcited by 22
  3. 2022Gendered Mental Health Stigma in Masked Language Modelscited by 17
  4. 2022On Robust Prefix-Tuning for Text Classificationcited by 15
  5. 2025How Deep Is Representational Bias in LLMs? The Cases of Caste and Religioncited by 14
  6. 2025HICode: Hierarchical Inductive Coding with LLMscited by 12
  7. 2024Classist Tools: Social Class Correlates with Performance in NLPcited by 9
  8. 2024How Are LLMs Mitigating Stereotyping Harms? Learning from Search Engine Studiescited by 9
  9. 2023GAIA Search: Hugging Face and Pyserini Interoperability for NLP Training Data Explorationcited by 8
  10. 2022Modeling Exemplification in Long-form Question Answering via Retrievalcited by 7
  11. 2022Measuring Harmful Representations in Scandinavian Language Modelscited by 7
  12. 2023Has It All Been Solved? Open NLP Research Questions Not Solved by Large Language Modelscited by 7
  13. 2023Concise Answers to Complex Questions: Summarization of Long-form Answerscited by 7
  14. 2023Evaluating AI systems under uncertain ground truth: a case study in dermatologycited by 7
  15. 2025When Cultures Meet: Multicultural Text-to-Image Generationcited by 7
  16. 2022Speaker Information Can Guide Models to Better Inductive Biases: A Case Study On Predicting Code-Switchingcited by 6
  17. 2023Dual Use Concerns of Generative AI and Large Language Modelscited by 6
  18. 2023Examining risks of racial biases in NLP tools for child protective servicescited by 6
  19. 2024Whose wife is it anyway? Assessing bias against same-gender relationships in machine translationcited by 6
  20. 2024Standard Language Ideology in AI-Generated Languagecited by 5
  21. 2025Identities are not Interchangeable: The Problem of Overgeneralization in Fair Machine Learningcited by 5
  22. 2022Undesirable Biases in NLP: Addressing Challenges of Measurementcited by 4
  23. 2023Measuring Intersectional Biases in Historical Documentscited by 4
  24. 2024A Comprehensive View of the Biases of Toxicity and Sentiment Analysis Methods Towards Utterances with African American English Expressionscited by 4
  25. 2024Creating an African American-Sounding TTS: Guidelines, Technical Challenges,and Surprising Evaluationscited by 4
  26. 2024Towards Algorithmic Fidelity: Mental Health Representation across Demographics in Synthetic vs. Human-generated Datacited by 4
  27. 2024Laissez-Faire Harms: Algorithmic Biases in Generative Language Modelscited by 4
  28. 2024On Tables with Numbers, with Numberscited by 4
  29. 2022Towards a Deep Multi-layered Dialectal Language Analysis: A Case Study of African-American Englishcited by 3
  30. 2022A Moral- and Event- Centric Inspection of Gender Bias in Fairy Tales at A Large Scalecited by 3
  31. 2023Using Semantic Similarity and Text Embedding to Measure the Social Media Echo of Strategic Communicationscited by 3
  32. 2023Cross-lingual Transfer Can Worsen Bias in Sentiment Analysiscited by 3
  33. 2023TalkUp: Paving the Way for Understanding Empowering Languagecited by 3
  34. 2024Ethical Concern Identification in NLP: A Corpus of ACL Anthology Ethics Statementscited by 3
  35. 2026Different Demographic Cues Yield Inconsistent Conclusions About LLM Personalization and Biascited by 3
  36. 2022A methodology to characterize bias and harmful stereotypes in natural language processing in Latin Americacited by 2
  37. 2023Conversation Modeling to Predict Derailmentcited by 2
  38. 2024GeniL: A Multilingual Dataset on Generalizing Languagecited by 2

50 most cited of 74 All 74 (JSON)