mlsanityData to 7 Oct 2026

Racial Bias in Hate Speech and Abusive Language Detection Datasets

arXiv 1905.12516 ↗First posted 29 May 2019cs.CL cs.LGcited by 208

Not on the map. Why?

Abstract (arXiv)

Technologies for abusive language detection are being developed and applied with little consideration of their potential biases. We examine racial bias in five different sets of Twitter data annotated for hate speech and abusive language. We train classifiers on these datasets and compare the predictions of these classifiers on tweets written in African-American English with those written in Standard American English. The results show evidence of systematic racial bias in all datasets, as classifiers trained on them tend to predict that tweets written in African-American English are abusive at substantially higher rates. If these abusive language detection systems are used in the field they will therefore have a disproportionate negative impact on African-American social media users. Consequently, these systems may discriminate against the groups who are often the targets of the abuse we are trying to detect.

Key facts. Cited by 208 papers in the map (data through 7 Oct 2026). First posted on arXiv 29 May 2019.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2021: 55; 2022: 44; 2023: 39; 2024: 35; 2025: 21; 2026: 14.

208 papers.

Sort:

By area:

  1. 2021BBQ: A Hand-Built Bias Benchmark for Question Answeringcited by 683
  2. 2021DExperts: Decoding-Time Controlled Text Generation with Experts and Anti-Expertscited by 434
  3. 2022A Holistic Approach to Undesired Content Detection in the Real Worldcited by 337
  4. 2021Dealing with Disagreements: Looking Beyond the Majority Vote in Subjective Annotationscited by 306
  5. 2021Annotators with Attitudes: How Annotator Beliefs And Identities Bias Toxic Language Detectioncited by 258
  6. 2024OR-Bench: An Over-Refusal Benchmark for Large Language Modelscited by 222
  7. 2021Latent Hatred: A Benchmark for Understanding Implicit Hate Speechcited by 216
  8. 2023Evaluating Large Language Models: A Comprehensive Surveycited by 200
  9. 2022On Second Thought, Let's Not Think Step by Step! Bias and Toxicity in Zero-Shot Reasoningcited by 191
  10. 2022A New Generation of Perspective API: Efficient Multilingual Character-level Transformerscited by 181
  11. 2021Challenges in Detoxifying Language Modelscited by 169
  12. 2023Should ChatGPT be Biased? Challenges and Risks of Bias in Large Language Modelscited by 144
38 more
  1. 2021Teach Me to Explain: A Review of Datasets for Explainable Natural Language Processingcited by 126
  2. 2021Understanding and Evaluating Racial Biases in Image Captioningcited by 108
  3. 2021Re-imagining Algorithmic Fairness in India and Beyondcited by 102
  4. 2021Challenges in Automated Debiasing for Toxic Language Detectioncited by 95
  5. 2021Competency Problems: On Finding and Removing Artifacts in Language Datacited by 91
  6. 2023NLPositionality: Characterizing Design Biases of Datasets and Modelscited by 91
  7. 2023Evaluating the Social Impact of Generative AI Systems in Systems and Societycited by 91
  8. 2021MOMENTA: A Multimodal Framework for Detecting Harmful Memes and Their Targetscited by 85
  9. 2021Quantifying Social Biases in NLP: A Generalization and Empirical Comparison of Extrinsic Fairness Metricscited by 83
  10. 2021On Measures of Biases and Harms in NLPcited by 81
  11. 2021A Survey of Race, Racism, and Anti-Racism in NLPcited by 74
  12. 2021On the Safety of Conversational Models: Taxonomy, Dataset, and Benchmarkcited by 73
  13. 2022Is Your Toxicity My Toxicity? Exploring the Impact of Rater Identity on Toxicity Annotationcited by 67
  14. 2021Towards generalisable hate speech detection: a review on obstacles and solutionscited by 66
  15. 2022Handling Bias in Toxic Speech Detection: A Surveycited by 60
  16. 2021Detecting Hate Speech with GPT-3cited by 55
  17. 2022Re-contextualizing Fairness in NLP: The Case of Indiacited by 55
  18. 2021Changing the World by Changing the Datacited by 51
  19. 2022Multilingual HateCheck: Functional Tests for Multilingual Hate Speech Detection Modelscited by 51
  20. 2022Multi-VALUE: A Framework for Cross-Dialectal English NLPcited by 51
  21. 2023On the Challenges of Using Black-Box APIs for Toxicity Evaluation in Researchcited by 44
  22. 2022Human-Machine Collaboration Approaches to Build a Dialogue Dataset for Hate Speech Counteringcited by 40
  23. 2021Overview of the HASOC track at FIRE 2020: Hate Speech and Offensive Content Identification in Indo-European Languagescited by 38
  24. 2021Disembodied Machine Learning: On the Illusion of Objectivity in NLPcited by 37
  25. 2022Towards Identifying Social Bias in Dialog Systems: Frame, Datasets, and Benchmarkscited by 36
  26. 2023Comparing Biases and the Impact of Multilingual Training across Multiple Languagescited by 36
  27. 2023Evaluation of African American Language Bias in Natural Language Generationcited by 36
  28. 2021Perceptual Score: What Data Modalities Does Your Model Perceive?cited by 35
  29. 2023COBRA Frames: Contextual Reasoning about Effects and Harms of Offensive Statementscited by 34
  30. 2024RTP-LX: Can LLMs Evaluate Toxicity in Multilingual Scenarios?cited by 34
  31. 2022Data Governance in the Age of Large-Scale Data-Driven Language Technologycited by 32
  32. 2021Detecting Cross-Geographic Biases in Toxicity Modeling on Social Mediacited by 31
  33. 2021Does Pretraining for Summarization Require Knowledge Transfer?cited by 30
  34. 2022Using Pre-Trained Language Models for Producing Counter Narratives Against Hate Speech: a Comparative Studycited by 30
  35. 2023HARE: Explainable Hate Speech Detection with Step-by-Step Reasoningcited by 30
  36. 2024ToxiCloakCN: Evaluating Robustness of Offensive Language Detection in Chinese with Cloaking Perturbationscited by 30
  37. 2022On Explaining Multimodal Hateful Meme Detection Modelscited by 29
  38. 2023Sociodemographic Bias in Language Models: A Survey and Forward Pathcited by 27

50 most cited of 208 All 208 (JSON)