# Agreeing to Disagree: Annotating Offensive Language Datasets with Annotators' Disagreement

arXiv: https://arxiv.org/abs/2109.13563 · HTML: https://mlsanity.com/paper/2109.13563/

## Key facts

- Cited by 72 papers in the map (data through 14 Oct 2026).
- Builds on 9 of its 37 references that are in the map.
- First posted on arXiv 28 Sep 2021.

## Builds on (most cited first, 9 of 9)

- [Automated Hate Speech Detection and the Problem of Offensive Language](https://mlsanity.com/paper/1703.04009/) (2017)
- [Social Bias Frames: Reasoning about Social and Power Implications of Language](https://mlsanity.com/paper/1911.03891/) (2019)
- [Predicting the Type and Target of Offensive Posts in Social Media](https://mlsanity.com/paper/1902.09666/) (2019)
- [Large Scale Crowdsourcing and Characterization of Twitter Abusive Behavior](https://mlsanity.com/paper/1802.00393/) (2018)
- [Human uncertainty makes classification more robust](https://mlsanity.com/paper/1908.07086/) (2019)
- [Sentence Encoders on STILTs: Supplementary Training on Intermediate Labeled-data Tasks](https://mlsanity.com/paper/1811.01088/) (2018)
- [SemEval-2020 Task 12: Multilingual Offensive Language Identification in Social Media (OffensEval 2020)](https://mlsanity.com/paper/2006.07235/) (2020)
- [Deep learning from crowds](https://mlsanity.com/paper/1709.01779/) (2017)
- [A Crowdsourced Frame Disambiguation Corpus with Ambiguity](https://mlsanity.com/paper/1904.06101/) (2019)

## Built on by (most cited first, 30 of 72)

- [ToxiGen: A Large-Scale Machine-Generated Dataset for Adversarial and Implicit Hate Speech Detection](https://mlsanity.com/paper/2203.09509/) (2022)
- [The 'Problem' of Human Label Variation: On Ground Truth in Data, Modeling and Evaluation](https://mlsanity.com/paper/2211.02570/) (2022)
- [SemEval-2023 Task 11: Learning With Disagreements (LeWiDi)](https://mlsanity.com/paper/2304.14803/) (2023)
- [When Do Annotator Demographics Matter? Measuring the Influence of Annotator Demographics with the POPQUORN Dataset](https://mlsanity.com/paper/2306.06826/) (2023)
- [You Are What You Annotate: Towards Better Models through Annotator Representations](https://mlsanity.com/paper/2305.14663/) (2023)
- [Capturing Perspectives of Crowdsourced Annotators in Subjective Learning Tasks](https://mlsanity.com/paper/2311.09743/) (2023)
- [The Effectiveness of LLMs as Annotators: A Comparative Overview and Empirical Analysis of Direct Representation](https://mlsanity.com/paper/2405.01299/) (2024)
- [Automatic Evaluation and Moderation of Open-domain Dialogue Systems](https://mlsanity.com/paper/2111.02110/) (2021)
- [Aligning Language Models with Human Preferences via a Bayesian Approach](https://mlsanity.com/paper/2310.05782/) (2023)
- [On Diversified Preferences of Large Language Model Alignment](https://mlsanity.com/paper/2312.07401/) (2023)
- [Consistency is Key: Disentangling Label Variation in Natural Language Processing with Intra-Annotator Agreement](https://mlsanity.com/paper/2301.10684/) (2023)
- [Unifying Data Perspectivism and Personalization: An Application to Social Norms](https://mlsanity.com/paper/2210.14531/) (2022)
- [Annotator-Centric Active Learning for Subjective NLP Tasks](https://mlsanity.com/paper/2404.15720/) (2024)
- [Vicarious Offense and Noise Audit of Offensive Speech Classifiers: Unifying Human and Machine Disagreement on What is Offensive](https://mlsanity.com/paper/2301.12534/) (2023)
- [Is LLM an Overconfident Judge? Unveiling the Capabilities of LLMs in Detecting Offensive Language with Annotation Disagreement](https://mlsanity.com/paper/2502.06207/) (2025)
- [Crowd-Calibrator: Can Annotator Disagreement Inform Calibration in Subjective Tasks?](https://mlsanity.com/paper/2408.14141/) (2024)
- [Two Contrasting Data Annotation Paradigms for Subjective NLP Tasks](https://mlsanity.com/paper/2112.07475/) (2021)
- [D3CODE: Disentangling Disagreements in Data across Cultures on Offensiveness Detection and Evaluation](https://mlsanity.com/paper/2404.10857/) (2024)
- [iLab at SemEval-2023 Task 11 Le-Wi-Di: Modelling Disagreement or Modelling Perspectives?](https://mlsanity.com/paper/2305.06074/) (2023)
- [Don't Blame the Data, Blame the Model: Understanding Noise and Bias When Learning from Subjective Annotations](https://mlsanity.com/paper/2403.04085/) (2024)
- [Subjective textitIsms? On the Danger of Conflating Hate and Offence in Abusive Language Detection](https://mlsanity.com/paper/2403.02268/) (2024)
- [LeWiDi-2025 at NLPerspectives: Third Edition of the Learning with Disagreements Shared Task](https://mlsanity.com/paper/2510.08460/) (2025)
- [Corpus Considerations for Annotator Modeling and Scaling](https://mlsanity.com/paper/2404.02340/) (2024)
- [Re-examining Sexism and Misogyny Classification with Annotator Attitudes](https://mlsanity.com/paper/2410.03543/) (2024)
- [Training and Evaluating with Human Label Variation: An Empirical Study](https://mlsanity.com/paper/2502.01891/) (2025)
- [Modeling subjectivity (by Mimicking Annotator Annotation) in toxic comment identification across diverse communities](https://mlsanity.com/paper/2311.00203/) (2023)
- [ARTICLE: Annotator Reliability Through In-Context Learning](https://mlsanity.com/paper/2409.12218/) (2024)
- ["Stop Asian Hate!" : Refining Detection of Anti-Asian Hate Speech During the COVID-19 Pandemic](https://mlsanity.com/paper/2112.02265/) (2021)
- [Understanding Interpersonal Conflict Types and their Impact on Perception Classification](https://mlsanity.com/paper/2208.08758/) (2022)
- [The State of Profanity Obfuscation in Natural Language Processing](https://mlsanity.com/paper/2210.07595/) (2022)

## Abstract (arXiv)

Since state-of-the-art approaches to offensive language detection rely on supervised learning, it is crucial to quickly adapt them to the continuously evolving scenario of social media. While several approaches have been proposed to tackle the problem from an algorithmic perspective, so to reduce the need for annotated data, less attention has been paid to the quality of these data. Following a trend that has emerged recently, we focus on the level of agreement among annotators while selecting data to create offensive language datasets, a task involving a high level of subjectivity. Our study comprises the creation of three novel datasets of English tweets covering different topics and having five crowd-sourced judgments each. We also present an extensive set of experiments showing that selecting training and test data according to different levels of annotators' agreement has a strong effect on classifiers performance and robustness. Our findings are further validated in cross-domain experiments and studied using a popular benchmark dataset. We show that such hard cases, where low agreement is present, are not necessarily due to poor-quality annotation and we advocate for a higher presence of ambiguous cases in future datasets, particularly in test sets, to better account for the different points of view expressed online.