# Ground-Truth, Whose Truth? -- Examining the Challenges with Annotating Toxic Text Datasets

arXiv: https://arxiv.org/abs/2112.03529 · HTML: https://mlsanity.com/paper/2112.03529/

## Key facts

- Cited by 11 papers in the map (data through 14 Oct 2026).
- Builds on 8 of its 28 references that are in the map.
- First posted on arXiv 7 Dec 2021.

## Builds on (most cited first, 8 of 8)

- [HateXplain: A Benchmark Dataset for Explainable Hate Speech Detection](https://mlsanity.com/paper/2012.10289/) (2020)
- [Pervasive Label Errors in Test Sets Destabilize Machine Learning Benchmarks](https://mlsanity.com/paper/2103.14749/) (2021)
- [Social Bias Frames: Reasoning about Social and Power Implications of Language](https://mlsanity.com/paper/1911.03891/) (2019)
- [Dealing with Disagreements: Looking Beyond the Majority Vote in Subjective Annotations](https://mlsanity.com/paper/2110.05719/) (2021)
- [Are We Modeling the Task or the Annotator? An Investigation of Annotator Bias in Natural Language Understanding Datasets](https://mlsanity.com/paper/1908.07898/) (2019)
- [What is the ground truth? Reliability of multi-annotator data for audio tagging](https://mlsanity.com/paper/2104.04214/) (2021)
- ["Garbage In, Garbage Out" Revisited: What Do Machine Learning Application Papers Report About Human-Labeled Training Data?](https://mlsanity.com/paper/2107.02278/) (2021)
- [Goldilocks: Consistent Crowdsourced Scalar Annotations with Relative Uncertainty](https://mlsanity.com/paper/2108.01799/) (2021)

## Built on by (most cited first, 11 of 11)

- [GRASP: A Disagreement Analysis Framework to Assess Group Associations in Perspectives](https://mlsanity.com/paper/2311.05074/) (2023)
- [Annotation Sensitivity: Training Data Collection Methods Affect Model Performance](https://mlsanity.com/paper/2311.14212/) (2023)
- [Detectors for Safe and Reliable LLMs: Implementations, Uses, and Limitations](https://mlsanity.com/paper/2403.06009/) (2024)
- [Critical Perspectives: A Benchmark Revealing Pitfalls in PerspectiveAPI](https://mlsanity.com/paper/2301.01874/) (2023)
- [Modeling subjectivity (by Mimicking Annotator Annotation) in toxic comment identification across diverse communities](https://mlsanity.com/paper/2311.00203/) (2023)
- [Whose Preferences? Differences in Fairness Preferences and Their Impact on the Fairness of AI Utilizing Human Feedback](https://mlsanity.com/paper/2406.05902/) (2024)
- [The Risks of Machine Learning Systems](https://mlsanity.com/paper/2204.09852/) (2022)
- [SoUnD Framework: Analyzing (So)cial Representation in (Un)structured (D)ata](https://mlsanity.com/paper/2311.17259/) (2023)
- [Beyond Agreement: Rethinking Ground Truth in Educational AI Annotation](https://mlsanity.com/paper/2508.00143/) (2025)
- [Combating Toxic Language: A Review of LLM-Based Strategies for Software Engineering](https://mlsanity.com/paper/2504.15439/) (2025)
- [Real-World AI Evaluation: How FRAME Generates Systematic Evidence to Resolve the Decision-Maker's Dilemma](https://mlsanity.com/paper/2603.13294/) (2026)

## Abstract (arXiv)

The use of machine learning (ML)-based language models (LMs) to monitor content online is on the rise. For toxic text identification, task-specific fine-tuning of these models are performed using datasets labeled by annotators who provide ground-truth labels in an effort to distinguish between offensive and normal content. These projects have led to the development, improvement, and expansion of large datasets over time, and have contributed immensely to research on natural language. Despite the achievements, existing evidence suggests that ML models built on these datasets do not always result in desirable outcomes. Therefore, using a design science research (DSR) approach, this study examines selected toxic text datasets with the goal of shedding light on some of the inherent issues and contributing to discussions on navigating these challenges for existing and future projects. To achieve the goal of the study, we re-annotate samples from three toxic text datasets and find that a multi-label approach to annotating toxic text samples can help to improve dataset quality. While this approach may not improve the traditional metric of inter-annotator agreement, it may better capture dependence on context and diversity in annotators. We discuss the implications of these results for both theory and practice.