mlsanityData to 7 Oct 2026

Challenges in Automated Debiasing for Toxic Language Detection

arXiv 2102.00086 ↗First posted 29 Jan 2021cs.CLcited by 95

Not on the map. Why?

Abstract (arXiv)

Biased associations have been a challenge in the development of classifiers for detecting toxic language, hindering both fairness and accuracy. As potential solutions, we investigate recently introduced debiasing methods for text classification datasets and models, as applied to toxic language detection. Our focus is on lexical (e.g., swear words, slurs, identity mentions) and dialectal markers (specifically African American English). Our comprehensive experiments establish that existing methods are limited in their ability to prevent biased behavior in current toxicity detectors. We then propose an automatic, dialect-aware data correction method, as a proof-of-concept. Despite the use of synthetic labels, this method reduces dialectal associations with toxicity. Overall, our findings show that debiasing a model trained on biased toxic language data is not as effective as simply relabeling the data to remove existing biases.

Key facts. Cited by 95 papers in the map (data through 7 Oct 2026). Builds on 31 of its 49 references that are in the map. First posted on arXiv 29 Jan 2021.

Builds on

31 of 49 references.

References · 31

  1. 2020Language Models are Few-Shot Learnerscited by 32,199
  2. 2019RoBERTa: A Robustly Optimized BERT Pretraining Approachcited by 13,266
  3. 2016SQuAD: 100,000+ Questions for Machine Comprehension of Textcited by 4,115
  4. 2017A Broad-Coverage Challenge Corpus for Sentence Understanding through Inferencecited by 2,665
  5. 2016Equality of Opportunity in Supervised Learningcited by 2,000
  6. 2015A large annotated corpus for learning natural language inferencecited by 1,837
  7. 2020RealToxicityPrompts: Evaluating Neural Toxic Degeneration in Language Modelscited by 1,235
  8. 2020Language (Technology) is Power: A Critical Survey of "Bias" in NLPcited by 951
  9. 2017Automated Hate Speech Detection and the Problem of Offensive Languagecited by 724
  10. 2018Mitigating Unwanted Biases with Adversarial Learningcited by 663
  11. 2018Annotation Artifacts in Natural Language Inference Datacited by 656
  12. 2020Dataset Cartography: Mapping and Diagnosing Datasets with Training Dynamicscited by 423
19 more
  1. 2019Social Bias Frames: Reasoning about Social and Power Implications of Languagecited by 377
  2. 2019Don't Take the Easy Way Out: Ensemble Based Methods for Avoiding Known Dataset Biasescited by 279
  3. 2018Large Scale Crowdsourcing and Characterization of Twitter Abusive Behaviorcited by 265
  4. 2019Racial Bias in Hate Speech and Abusive Language Detection Datasetscited by 208
  5. 2016Demographic Dialectal Variation in Social Media: A Case Study of African-American Englishcited by 194
  6. 2019Build it Break it Fix it for Dialogue Safety: Robustness from Adversarial Human Attackcited by 176
  7. 2020Adversarial Filters of Dataset Biasescited by 159
  8. 2018Reducing Gender Bias in Abusive Language Detectioncited by 152
  9. 2019End-to-End Bias Mitigation by Modelling Biases in Corporacited by 127
  10. 2019Unlearn Dataset Bias in Natural Language Inference by Fitting the Residualcited by 119
  11. 2019Lessons from Archives: Strategies for Collecting Sociocultural Data in Machine Learningcited by 116
  12. 2017Measuring the Reliability of Hate Speech Annotations: The Case of the European Refugee Crisiscited by 95
  13. 2020Hate Speech Detection and Racial Bias Mitigation in Social Media based on BERT modelcited by 87
  14. 2020Mind the Trade-off: Debiasing NLU Models without Degrading the In-distribution Performancecited by 85
  15. 2020Demoting Racial Bias in Hate Speech Detectioncited by 62
  16. 2020Investigating African-American Vernacular English in Transformer-Based Text Generationcited by 52
  17. 2017The Effect of Different Writing Tasks on Linguistic Style: A Case Study of the ROC Story Cloze Taskcited by 39
  18. 2020Intersectional Bias in Hate Speech and Abusive Language Datasetscited by 22
  19. 2019Empirical Analysis of Multi-Task Learning for Reducing Model Bias in Toxic Comment Detectioncited by 6

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2021: 20; 2022: 23; 2023: 22; 2024: 19; 2025: 6; 2026: 5.

95 papers.

Sort:

By area:

  1. 2022ToxiGen: A Large-Scale Machine-Generated Dataset for Adversarial and Implicit Hate Speech Detectioncited by 419
  2. 2023SOTOPIA: Interactive Evaluation for Social Intelligence in Language Agentscited by 350
  3. 2021Dealing with Disagreements: Looking Beyond the Majority Vote in Subjective Annotationscited by 306
  4. 2021Annotators with Attitudes: How Annotator Beliefs And Identities Bias Toxic Language Detectioncited by 258
  5. 2024OR-Bench: An Over-Refusal Benchmark for Large Language Modelscited by 222
  6. 2021Challenges in Detoxifying Language Modelscited by 169
  7. 2023Having Beer after Prayer? Measuring Cultural Bias in Large Language Modelscited by 153
  8. 2023Mechanistically analyzing the effects of fine-tuning on procedurally defined taskscited by 110
  9. 2022ProsocialDialog: A Prosocial Backbone for Conversational Agentscited by 109
  10. 2021Detoxifying Language Models Risks Marginalizing Minority Voicescited by 106
  11. 2021Time Waits for No One! Analysis and Challenges of Temporal Misalignmentcited by 91
  12. 2023A Survey on Fairness in Large Language Modelscited by 91
38 more
  1. 2025GuardReasoner: Towards Reasoning-based LLM Safeguardscited by 89
  2. 2021MOMENTA: A Multimodal Framework for Detecting Harmful Memes and Their Targetscited by 85
  3. 2021Anticipating Safety Issues in E2E Conversational AI: Framework and Toolingcited by 77
  4. 2022Handling Bias in Toxic Speech Detection: A Surveycited by 60
  5. 2024Is this the real life? Is this just fantasy? The Misleading Success of Simulating Social Interactions With LLMscited by 59
  6. 2025Safety in Large Reasoning Models: A Surveycited by 58
  7. 2021Just Say No: Analyzing the Stance of Neural Dialogue Generation in Offensive Contextscited by 56
  8. 2022Entropy-based Attention Regularization Frees Unintended Bias Mitigation from Listscited by 54
  9. 2022Multi-VALUE: A Framework for Cross-Dialectal English NLPcited by 51
  10. 2023Facilitating Fine-grained Detection of Chinese Toxic Language: Hierarchical Taxonomy, Resources, and Benchmarkscited by 45
  11. 2024R2-Guard: Robust Reasoning Enabled LLM Guardrail via Knowledge-Enhanced Logical Reasoningcited by 41
  12. 2021General-Purpose Question-Answering with Macawcited by 40
  13. 2022Mitigating Gender Bias in Distilled Language Models via Counterfactual Role Reversalcited by 40
  14. 2022Towards Identifying Social Bias in Dialog Systems: Frame, Datasets, and Benchmarkscited by 36
  15. 2023Evaluation of African American Language Bias in Natural Language Generationcited by 36
  16. 2023COBRA Frames: Contextual Reasoning about Effects and Harms of Offensive Statementscited by 34
  17. 2022Unified Detoxifying and Debiasing in Language Generation via Inference-time Adaptive Optimizationcited by 32
  18. 2021Detecting Cross-Geographic Biases in Toxicity Modeling on Social Mediacited by 31
  19. 2022On Explaining Multimodal Hateful Meme Detection Modelscited by 29
  20. 2024MLLM-as-a-Judge for Image Safety without Human Labelingcited by 23
  21. 2024Adversarial Attacks of Vision Tasks in the Past 10 Years: A Surveycited by 22
  22. 2025Rejected Dialects: Biases Against African American Language in Reward Modelscited by 20
  23. 2025From Judgment to Interference: Early Stopping LLM Harmful Outputs via Streaming Content Monitoringcited by 20
  24. 2021Mitigating Racial Biases in Toxic Language Detection with an Equity-Based Ensemble Frameworkcited by 19
  25. 2024NLP Systems That Can't Tell Use from Mention Censor Counterspeech, but Teaching the Distinction Helpscited by 19
  26. 2021Measuring and Improving Model-Moderator Collaboration using Uncertainty Estimationcited by 18
  27. 2022Controlling Bias Exposure for Fair Interpretable Predictionscited by 18
  28. 2021LMMS Reloaded: Transformer-based Sense Embeddings for Disambiguation and Beyondcited by 17
  29. 2024Akal Badi ya Bias: An Exploratory Study of Gender Bias in Hindi Language Technologycited by 17
  30. 2024PolygloToxicityPrompts: Multilingual Evaluation of Neural Toxic Degeneration in Large Language Modelscited by 16
  31. 2023On the Trustworthiness Landscape of State-of-the-art Generative Models: A Survey and Outlookcited by 15
  32. 2022Improving Generalizability in Implicitly Abusive Language Detection with Concept Activation Vectorscited by 14
  33. 2022Flexible text generation for counterfactual fairness probingcited by 13
  34. 2023Towards Agile Text Classifiers for Everyonecited by 13
  35. 2021Style Pooling: Automatic Text Style Obfuscation for Improved Classification Fairnesscited by 12
  36. 2023DADA: Dialect Adaptation via Dynamic Aggregation of Linguistic Rulescited by 12
  37. 2024Algorithmic Arbitrariness in Content Moderationcited by 12
  38. 2025Lost in Moderation: How Commercial Content Moderation APIs Over- and Under-Moderate Group-Targeted Hate Speech and Linguistic Variationscited by 12

50 most cited of 95 All 95 (JSON)