mlsanityData to 7 Oct 2026

On Measures of Biases and Harms in NLP

arXiv 2108.03362 ↗First posted 7 Aug 2021cs.CL cs.CYcited by 81

Not on the map. Why?

Abstract (arXiv)

Recent studies show that Natural Language Processing (NLP) technologies propagate societal biases about demographic groups associated with attributes such as gender, race, and nationality. To create interventions and mitigate these biases and associated harms, it is vital to be able to detect and measure such biases. While existing works propose bias evaluation and mitigation methods for various tasks, there remains a need to cohesively understand the biases and the specific harms they measure, and how different measures compare with each other. To address this gap, this work presents a practical framework of harms and a series of questions that practitioners can answer to guide the development of bias measures. As a validation of our framework and documentation questions, we also present several case studies of how existing bias measures in NLP -- both intrinsic measures of bias in representations and extrinsic measures of bias of downstream applications -- can be aligned with different harms and how our proposed documentation questions facilitates more holistic understanding of what bias measures are measuring.

Key facts. Cited by 81 papers in the map (data through 7 Oct 2026). Builds on 55 of its 116 references that are in the map. First posted on arXiv 7 Aug 2021.

Builds on

55 of 116 references.

References · 55

  1. 2018Datasheets for Datasetscited by 1,371
  2. 2016Man is to Computer Programmer as Woman is to Homemaker? Debiasing Word Embeddingscited by 1,304
  3. 2018Model Cards for Model Reportingcited by 1,068
  4. 2020Language (Technology) is Power: A Critical Survey of "Bias" in NLPcited by 951
  5. 2020StereoSet: Measuring stereotypical bias in pretrained language modelscited by 911
  6. 2016Semantics derived automatically from language corpora contain human-like biasescited by 894
  7. 2017Automated Hate Speech Detection and the Problem of Offensive Languagecited by 724
  8. 2020CrowS-Pairs: A Challenge Dataset for Measuring Social Biases in Masked Language Modelscited by 703
  9. 2018Gender Bias in Coreference Resolution: Evaluation and Debiasing Methodscited by 657
  10. 2019The Woman Worked as a Babysitter: On Biases in Language Generationcited by 540
  11. 2020Null It Out: Guarding Protected Attributes by Iterative Nullspace Projectioncited by 442
  12. 2018Gender Bias in Coreference Resolutioncited by 438
38 more
  1. 2021BOLD: Dataset and Metrics for Measuring Biases in Open-Ended Language Generationcited by 410
  2. 2019Nuanced Metrics for Measuring Unintended Bias with Real Data for Text Classificationcited by 396
  3. 2019Social Bias Frames: Reasoning about Social and Power Implications of Languagecited by 377
  4. 2019Bias in Bios: A Case Study of Semantic Representation Bias in a High-Stakes Settingcited by 346
  5. 2019Measurement and Fairnesscited by 299
  6. 2019Mitigating Gender Bias in Natural Language Processing: Literature Reviewcited by 287
  7. 2019Lipstick on a Pig: Debiasing Methods Cover up Systematic Gender Biases in Word Embeddings But do not Remove Themcited by 268
  8. 2018Large Scale Crowdsourcing and Characterization of Twitter Abusive Behaviorcited by 265
  9. 2019Evaluating Gender Bias in Machine Translationcited by 247
  10. 2018Examining Gender and Race Bias in Two Hundred Sentiment Analysis Systemscited by 234
  11. 2018Gender Bias in Neural Natural Language Processingcited by 210
  12. 2019Racial Bias in Hate Speech and Abusive Language Detection Datasetscited by 208
  13. 2019Gender Bias in Contextualized Word Embeddingscited by 203
  14. 2021Societal Biases in Language Generation: Progress and Challengescited by 201
  15. 2019Counterfactual Data Augmentation for Mitigating Gender Stereotypes in Languages with Rich Morphologycited by 200
  16. 2016Demographic Dialectal Variation in Social Media: A Case Study of African-American Englishcited by 194
  17. 2020Social Biases in NLP Models as Barriers for Persons with Disabilitiescited by 182
  18. 2019Build it Break it Fix it for Dialogue Safety: Robustness from Adversarial Human Attackcited by 176
  19. 2018Reducing Gender Bias in Abusive Language Detectioncited by 152
  20. 2019Queens are Powerful too: Mitigating Gender Bias in Dialogue Generationcited by 152
  21. 2019Reducing Sentiment Bias in Language Models via Counterfactual Evaluationcited by 148
  22. 2020Intrinsic Bias Metrics Do Not Correlate with Application Biascited by 148
  23. 2020UnQovering Stereotyping Biases via Underspecified Questionscited by 124
  24. 2019On Measuring and Mitigating Biased Inferences of Word Embeddingscited by 112
  25. 2019Does Gender Matter? Towards Fairness in Dialogue Systemscited by 89
  26. 2019Toward Gender-Inclusive Coreference Resolutioncited by 84
  27. 2020Contextualizing Hate Speech Classifiers with Post-hoc Explanationcited by 84
  28. 2018Event2Mind: Commonsense Inference on Events, Intents, and Reactionscited by 82
  29. 2019Perturbation Sensitivity Analysis to Detect Unintended Model Biasescited by 74
  30. 2019Attenuating Bias in Word Vectorscited by 64
  31. 2020Demoting Racial Bias in Hate Speech Detectioncited by 62
  32. 2020Gender Bias in Multilingual Embeddings and Cross-Lingual Transfercited by 55
  33. 2020Investigating African-American Vernacular English in Transformer-Based Text Generationcited by 52
  34. 2021Ethical-Advice Taker: Do Language Models Understand Natural Language Interventions?cited by 46
  35. 2020Mitigating Gender Bias for Neural Dialogue Generation with Adversarial Learningcited by 44
  36. 2020OSCaR: Orthogonal Subspace Correction and Rectification of Biases in Word Embeddingscited by 40
  37. 2020"You are grounded!": Latent Name Artifacts in Pre-trained Language Modelscited by 39
  38. 2021Lawyers are Dishonest? Quantifying Representational Harms in Commonsense Knowledge Resourcescited by 30

50 most cited of 55 All 55 (JSON)

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2022: 3; 2023: 21; 2024: 34; 2025: 17; 2026: 6.

81 papers.

Sort:

By area:

  1. 2022Scaling Instruction-Finetuned Language Modelscited by 2,997
  2. 2023PaLM 2 Technical Reportcited by 1,154
  3. 2023"Kelly is a Warm Person, Joseph is a Role Model": Gender Biases in LLM-Generated Reference Letterscited by 196
  4. 2023Do All Languages Cost the Same? Tokenization in the Era of Commercial Language Modelscited by 186
  5. 2023Language Model Behavior: A Comprehensive Surveycited by 92
  6. 2024Bias in Large Language Models: Origin, Evaluation, and Mitigationcited by 77
  7. 2024Fairness in Large Language Models: A Taxonomic Surveycited by 57
  8. 2023"I'm fully who I am": Towards Centering Transgender and Non-Binary Voices to Measure Biases in Open Language Generationcited by 56
  9. 2023Nationality Bias in Text Generationcited by 50
  10. 2023Are Personalized Stochastic Parrots More Dangerous? Evaluating Persona Biases in Dialogue Systemscited by 37
  11. 2024MBBQ: A Dataset for Cross-Lingual Comparison of Stereotypes in Generative LLMscited by 32
  12. 2024Large Language Models Portray Socially Subordinate Groups as More Homogeneous, Consistent with a Bias Observed in Humanscited by 31
38 more
  1. 2024A Robot Walks into a Bar: Can Language Models Serve as Creativity Support Tools for Comedy? An Evaluation of LLMs' Humour Alignment with Comedianscited by 29
  2. 2025Value Profiles for Encoding Human Variationcited by 29
  3. 2023Sociodemographic Bias in Language Models: A Survey and Forward Pathcited by 27
  4. 2024Towards interactive evaluations for interaction harms in human-AI systemscited by 27
  5. 2024Culturally Aware and Adapted NLP: A Taxonomy and a Survey of the State of the Artcited by 26
  6. 2023Building Socio-culturally Inclusive Stereotype Resources with Community Engagementcited by 25
  7. 2024Do Generative AI Models Output Harm while Representing Non-Western Cultures: Evidence from A Community-Centered Approachcited by 25
  8. 2024"They are uncultured": Unveiling Covert Harms and Social Threats in LLM Generated Conversationscited by 24
  9. 2025Reading Between the Prompts: How Stereotypes Shape LLM's Implicit Personalizationcited by 24
  10. 2023NBIAS: A Natural Language Processing Framework for Bias Identification in Textcited by 23
  11. 2023Identifying and Adapting Transformer-Components Responsible for Gender Bias in an English Language Modelcited by 22
  12. 2024Akal Badi ya Bias: An Exploratory Study of Gender Bias in Hindi Language Technologycited by 17
  13. 2023Unmasking Nationality Bias: A Study of Human Perception of Nationalities in AI-Generated Articlescited by 16
  14. 2023Evaluating Gender Bias of Pre-trained Language Models in Natural Language Inference by Considering All Labelscited by 16
  15. 2023Safety and Fairness for Content Moderation in Generative Modelscited by 15
  16. 2024What the Harm? Quantifying the Tangible Impact of Gender Bias in Machine Translation with a Human-centered Studycited by 15
  17. 2024From Melting Pots to Misrepresentations: Exploring Harms in Generative AIcited by 13
  18. 2024The Sociolinguistic Foundations of Language Modelingcited by 13
  19. 2023The Sentiment Problem: A Critical Survey towards Deconstructing Sentiment Analysiscited by 12
  20. 2024Distilling Text Style Transfer With Self-Explanation From LLMscited by 12
  21. 2024The Call for Socially Aware Language Technologiescited by 12
  22. 2024The Responsible Foundation Model Development Cheatsheet: A Review of Tools & Resourcescited by 11
  23. 2023Towards a Holistic Approach: Understanding Sociodemographic Biases in NLP Models using an Interdisciplinary Lenscited by 9
  24. 2023Comparing Intrinsic Gender Bias Evaluation Measures without using Human Annotated Examplescited by 8
  25. 2024RAcQUEt: Unveiling the Dangers of Overlooked Referential Ambiguity in Visual LLMscited by 8
  26. 2025A Tale of Two Identities: An Ethical Audit of Human and AI-Crafted Personascited by 8
  27. 2022Trustworthy Social Bias Measurementcited by 7
  28. 2023Fairness Evaluation in Text Classification: Machine Learning Practitioner Perspectives of Individual and Group Fairnesscited by 7
  29. 2024CLIMB: A Benchmark of Clinical Bias in Large Language Modelscited by 7
  30. 2025Taxonomizing Representational Harms using Speech Act Theorycited by 7
  31. 2024SPAR: Personalized Content-Based Recommendation via Long Engagement Attentioncited by 6
  32. 2024Navigating the Landscape of Hint Generation Research: From the Past to the Futurecited by 6
  33. 2024Investigating Annotator Bias in Large Language Models for Hate Speech Detectioncited by 6
  34. 2024Unveiling and Mitigating Bias in Large Language Model Recommendations: A Path to Fairnesscited by 6
  35. 2024Hey GPT, Can You be More Racist? Analysis from Crowdsourced Attempts to Elicit Biased Content from Generative AIcited by 6
  36. 2024Vision-Language Models Generate More Homogeneous Stories for Phenotypically Black Individualscited by 6
  37. 2025All Claims Are Equal, but Some Claims Are More Equal Than Others: Importance-Sensitive Factuality Evaluation of LLM Generationscited by 6
  38. 2024Cross-Lingual Transfer of Debiasing and Detoxification in Multilingual LLMs: An Extensive Investigationcited by 5

50 most cited of 81 All 81 (JSON)