# A Taxonomy of Rater Disagreements: Surveying Challenges & Opportunities from the Perspective of Annotating Online Toxicity

arXiv: https://arxiv.org/abs/2311.04345 · HTML: https://mlsanity.com/paper/2311.04345/

## Key facts

- Cited by 4 papers in the map (data through 14 Oct 2026).
- Builds on 23 of its 86 references that are in the map.
- First posted on arXiv 7 Nov 2023.

## Builds on (most cited first, 23 of 23)

- [Dealing with Disagreements: Looking Beyond the Majority Vote in Subjective Annotations](https://mlsanity.com/paper/2110.05719/) (2021)
- [A New Generation of Perspective API: Efficient Multilingual Character-level Transformers](https://mlsanity.com/paper/2202.11176/) (2022)
- [When the Majority is Wrong: Modeling Annotator Disagreement for Subjective Tasks](https://mlsanity.com/paper/2305.06626/) (2023)
- [Hate Speech Detection and Racial Bias Mitigation in Social Media based on BERT model](https://mlsanity.com/paper/2008.06460/) (2020)
- [Is Your Toxicity My Toxicity? Exploring the Impact of Rater Identity on Toxicity Annotation](https://mlsanity.com/paper/2205.00501/) (2022)
- [Designing Toxic Content Classification for a Diversity of Perspectives](https://mlsanity.com/paper/2106.04511/) (2021)
- [Spectral Methods meet EM: A Provably Optimal Algorithm for Crowdsourcing](https://mlsanity.com/paper/1406.3824/) (2014)
- [Handling Bias in Toxic Speech Detection: A Survey](https://mlsanity.com/paper/2202.00126/) (2022)
- [Challenges for Toxic Comment Classification: An In-Depth Error Analysis](https://mlsanity.com/paper/1809.07572/) (2018)
- [Whose Ground Truth? Accounting for Individual and Collective Identities Underlying Dataset Annotation](https://mlsanity.com/paper/2112.04554/) (2021)
- [Hate Speech and Counter Speech Detection: Conversational Context Does Matter](https://mlsanity.com/paper/2206.06423/) (2022)
- [Consistency is Key: Disentangling Label Variation in Natural Language Processing with Intra-Annotator Agreement](https://mlsanity.com/paper/2301.10684/) (2023)
- [Towards Robust Toxic Content Classification](https://mlsanity.com/paper/1912.06872/) (2019)
- [Abuse is Contextual, What about NLP? The Role of Context in Abusive Language Annotation and Detection](https://mlsanity.com/paper/2103.14916/) (2021)
- [The Origin and Value of Disagreement Among Data Labelers: A Case Study of the Individual Difference in Hate Speech Annotation](https://mlsanity.com/paper/2112.04030/) (2021)
- [Neural Character-based Composition Models for Abuse Detection](https://mlsanity.com/paper/1809.00378/) (2018)
- [Leveraging Multilingual Transformers for Hate Speech Detection](https://mlsanity.com/paper/2101.03207/) (2021)
- [Revisiting Contextual Toxicity Detection in Conversations](https://mlsanity.com/paper/2111.12447/) (2021)
- [Explaining with Examples: Lessons Learned from Crowdsourced Introductory Description of Information Visualizations](https://mlsanity.com/paper/2112.12364/) (2021)
- [Toxicity Detection can be Sensitive to the Conversational Context](https://mlsanity.com/paper/2111.10223/) (2021)
- [Reliable Crowdsourcing for Multi-Class Labeling using Coding Theory](https://mlsanity.com/paper/1309.3330/) (2013)
- [Understanding Toxicity Triggers on Reddit in the Context of Singapore](https://mlsanity.com/paper/2204.08806/) (2022)
- [Beyond Plain Toxic: Detection of Inappropriate Statements on Flammable Topics for the Russian Language](https://mlsanity.com/paper/2203.02392/) (2022)

## Built on by (most cited first, 4 of 4)

- [A Collaborative Content Moderation Framework for Toxicity Detection based on Conformalized Estimates of Annotation Disagreement](https://mlsanity.com/paper/2411.04090/) (2024)
- [Who and What? Using Linguistic Features and Annotator Characteristics to Analyze Annotation Variation](https://mlsanity.com/paper/2605.06318/) (2026)
- [The Unappreciated Role of Intent in Algorithmic Moderation of Social Media Content](https://mlsanity.com/paper/2405.11030/) (2024)
- [FECT: Factuality Evaluation of Interpretive AI-Generated Claims in Contact Center Conversation Transcripts](https://mlsanity.com/paper/2508.00889/) (2025)

## Abstract (arXiv)

Toxicity is an increasingly common and severe issue in online spaces. Consequently, a rich line of machine learning research over the past decade has focused on computationally detecting and mitigating online toxicity. These efforts crucially rely on human-annotated datasets that identify toxic content of various kinds in social media texts. However, such annotations historically yield low inter-rater agreement, which was often dealt with by taking the majority vote or other such approaches to arrive at a single ground truth label. Recent research has pointed out the importance of accounting for the subjective nature of this task when building and utilizing these datasets, and this has triggered work on analyzing and better understanding rater disagreements, and how they could be effectively incorporated into the machine learning developmental pipeline. While these efforts are filling an important gap, there is a lack of a broader framework about the root causes of rater disagreement, and therefore, we situate this work within that broader landscape. In this survey paper, we analyze a broad set of literature on the reasons behind rater disagreements focusing on online toxicity, and propose a detailed taxonomy for the same. Further, we summarize and discuss the potential solutions targeting each reason for disagreement. We also discuss several open issues, which could promote the future development of online toxicity research.