mlsanityData to 14 Oct 2026

Detecting Unintended Social Bias in Toxic Language Datasets

arXiv 2210.11762 ↗First posted 21 Oct 2022cs.CLcited by 12

Not on the map. Why?

Abstract (arXiv)

With the rise of online hate speech, automatic detection of Hate Speech, Offensive texts as a natural language processing task is getting popular. However, very little research has been done to detect unintended social bias from these toxic language datasets. This paper introduces a new dataset ToxicBias curated from the existing dataset of Kaggle competition named "Jigsaw Unintended Bias in Toxicity Classification". We aim to detect social biases, their categories, and targeted groups. The dataset contains instances annotated for five different bias categories, viz., gender, race/ethnicity, religion, political, and LGBTQ. We train transformer-based models using our curated datasets and report baseline performance for bias identification, target generation, and bias implications. Model biases and their mitigation are also discussed in detail. Our study motivates a systematic extraction of social bias data from toxic language datasets. All the codes and dataset used for experiments in this work are publicly available

Key facts. Cited by 12 papers in the map (data through 14 Oct 2026). Builds on 24 of its 54 references that are in the map. First posted on arXiv 21 Oct 2022.

Builds on

24 of 54 references.

References · 24

  1. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,399
  2. 2016Man is to Computer Programmer as Woman is to Homemaker? Debiasing Word Embeddingscited by 1,312
  3. 2020StereoSet: Measuring stereotypical bias in pretrained language modelscited by 924
  4. 2016Semantics derived automatically from language corpora contain human-like biasescited by 901
  5. 2017Automated Hate Speech Detection and the Problem of Offensive Languagecited by 736
  6. 2020CrowS-Pairs: A Challenge Dataset for Measuring Social Biases in Masked Language Modelscited by 710
  7. 2015Bidirectional LSTM-CRF Models for Sequence Taggingcited by 546
  8. 2022ToxiGen: A Large-Scale Machine-Generated Dataset for Adversarial and Implicit Hate Speech Detectioncited by 422
  9. 2019Nuanced Metrics for Measuring Unintended Bias with Real Data for Text Classificationcited by 404
  10. 2019On Measuring Social Biases in Sentence Encoderscited by 398
  11. 2019Social Bias Frames: Reasoning about Social and Power Implications of Languagecited by 380
  12. 2019Measuring Bias in Contextualized Word Representationscited by 276
12 more
  1. 2018Large Scale Crowdsourcing and Characterization of Twitter Abusive Behaviorcited by 270
  2. 2019Racial Bias in Hate Speech and Abusive Language Detection Datasetscited by 209
  3. 2018Reducing Gender Bias in Abusive Language Detectioncited by 152
  4. 2019Black is to Criminal as Caucasian is to Police: Detecting and Removing Multiclass Bias in Word Embeddingscited by 147
  5. 2019CONAN -- COunter NArratives through Nichesourcing: a Multilingual Dataset of Responses to Fight Online Hate Speechcited by 110
  6. 2020Hate Speech Detection and Racial Bias Mitigation in Social Media based on BERT modelcited by 87
  7. 2020Unmasking Contextual Stereotypes: Measuring and Mitigating BERT's Gender Biascited by 82
  8. 2021Human-in-the-Loop for Data Collection: a Multi-Target Counter Narrative Dataset to Fight Online Hate Speechcited by 82
  9. 2020Demoting Racial Bias in Hate Speech Detectioncited by 62
  10. 2020Intersectional Bias in Hate Speech and Abusive Language Datasetscited by 22
  11. 2020Reducing Unintended Identity Bias in Russian Hate Speech Detectioncited by 10
  12. 2022Hollywood Identity Bias Dataset: A Context Oriented Bias Analysis of Movie Dialoguescited by 5

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2023: 2; 2024: 5; 2025: 4; 2026: 1.

12 papers.

  1. 2024IndiBias: A Benchmark Dataset to Measure Social Biases in Language Models for Indian Contextcited by 32
  2. 2024NLP Systems That Can't Tell Use from Mention Censor Counterspeech, but Teaching the Distinction Helpscited by 20
  3. 2023GRASP: A Disagreement Analysis Framework to Assess Group Associations in Perspectivescited by 16
  4. 2024Large Language Model Bias Mitigation from the Perspective of Knowledge Editingcited by 16
  5. 2024Identifying and Mitigating Social Bias Knowledge in Language Modelscited by 13
  6. 2023Rethinking Multimodal Content Moderation from an Asymmetric Angle with Mixed-modalitycited by 9
  7. 2024On the Role of Speech Data in Reducing Toxicity Detection Biascited by 2
  8. 2025Investigating the Capabilities and Limitations of Machine Learning for Identifying Bias in English Language Data with Information and Heritage Professionalscited by 0
  9. 2025Stereotype Detection as a Catalyst for Enhanced Bias Detection: A Multi-Task Learning Approachcited by 0
  10. 2025Bridging Fairness and Explainability: Can Input-Based Explanations Promote Fairness in Hate Speech Detection?cited by 0
  11. 2025Open-DeBias: Toward Mitigating Open-Set Bias in Language Modelscited by 0
  12. 2026MPSelectTune: Prompt-type Selection for Fine-tuning improves Concept Unlearning in LLMscited by 0