# Hate Speech Criteria: A Modular Approach to Task-Specific Hate Speech Definitions

arXiv: https://arxiv.org/abs/2206.15455 · HTML: https://mlsanity.com/paper/2206.15455/

## Key facts

- Cited by 18 papers in the map (data through 14 Oct 2026).
- Builds on 22 of its 59 references that are in the map.
- First posted on arXiv 30 Jun 2022.

## Builds on (most cited first, 22 of 22)

- [Automated Hate Speech Detection and the Problem of Offensive Language](https://mlsanity.com/paper/1703.04009/) (2017)
- [HateXplain: A Benchmark Dataset for Explainable Hate Speech Detection](https://mlsanity.com/paper/2012.10289/) (2020)
- [Large Scale Crowdsourcing and Characterization of Twitter Abusive Behavior](https://mlsanity.com/paper/1802.00393/) (2018)
- [Latent Hatred: A Benchmark for Understanding Implicit Hate Speech](https://mlsanity.com/paper/2109.05322/) (2021)
- [HateCheck: Functional Tests for Hate Speech Detection Models](https://mlsanity.com/paper/2012.15606/) (2020)
- [Racial Bias in Hate Speech and Abusive Language Detection Datasets](https://mlsanity.com/paper/1905.12516/) (2019)
- [Hate Speech Dataset from a White Supremacy Forum](https://mlsanity.com/paper/1809.04444/) (2018)
- [MultiVerS: Improving scientific claim verification with weak supervision and full-document context](https://mlsanity.com/paper/2112.01640/) (2021)
- [Directions in Abusive Language Training Data: Garbage In, Garbage Out](https://mlsanity.com/paper/2004.01670/) (2020)
- [Multilingual and Multi-Aspect Hate Speech Analysis](https://mlsanity.com/paper/1908.11049/) (2019)
- [A Benchmark Dataset for Learning to Intervene in Online Hate Speech](https://mlsanity.com/paper/1909.04251/) (2019)
- [Understanding Abuse: A Typology of Abusive Language Detection Subtasks](https://mlsanity.com/paper/1705.09899/) (2017)
- [CONAN -- COunter NArratives through Nichesourcing: a Multilingual Dataset of Responses to Fight Online Hate Speech](https://mlsanity.com/paper/1910.03270/) (2019)
- [Human-in-the-Loop for Data Collection: a Multi-Target Counter Narrative Dataset to Fight Online Hate Speech](https://mlsanity.com/paper/2107.08720/) (2021)
- [Detecting Online Hate Speech Using Context Aware Models](https://mlsanity.com/paper/1710.07395/) (2017)
- [Characterizing and Detecting Hateful Users on Twitter](https://mlsanity.com/paper/1803.08977/) (2018)
- [Hatemoji: A Test Suite and Adversarially-Generated Dataset for Benchmarking and Detecting Emoji-based Hate](https://mlsanity.com/paper/2108.05921/) (2021)
- [Confronting Abusive Language Online: A Survey from the Ethical and Human Rights Perspective](https://mlsanity.com/paper/2012.12305/) (2020)
- [Peer to Peer Hate: Hate Speech Instigators and Their Targets](https://mlsanity.com/paper/1804.04649/) (2018)
- [A Legal Approach to Hate Speech: Operationalizing the EU's Legal Framework against the Expression of Hatred as an NLP Task](https://mlsanity.com/paper/2004.03422/) (2020)
- [Are Chess Discussions Racist? An Adversarial Hate Speech Data Set](https://mlsanity.com/paper/2011.10280/) (2020)
- [On Analyzing Annotation Consistency in Online Abusive Behavior Datasets](https://mlsanity.com/paper/2006.13507/) (2020)

## Built on by (most cited first, 18 of 18)

- [COBRA Frames: Contextual Reasoning about Effects and Harms of Offensive Statements](https://mlsanity.com/paper/2306.01985/) (2023)
- [Improving Adversarial Data Collection by Supporting Annotators: Lessons from GAHD, a German Hate Speech Dataset](https://mlsanity.com/paper/2403.19559/) (2024)
- [Crowd-Calibrator: Can Annotator Disagreement Inform Calibration in Subjective Tasks?](https://mlsanity.com/paper/2408.14141/) (2024)
- [DefVerify: Do Hate Speech Models Reflect Their Dataset's Definition?](https://mlsanity.com/paper/2410.15911/) (2024)
- [A Modular Taxonomy for Hate Speech Definitions and Its Impact on Zero-Shot LLM Classification Performance](https://mlsanity.com/paper/2506.18576/) (2025)
- [Untangling Hate Speech Definitions: A Semantic Componential Analysis Across Cultures and Domains](https://mlsanity.com/paper/2411.07417/) (2024)
- [Factoring Hate Speech: A New Annotation Framework to Study Hate Speech in Social Media](https://mlsanity.com/paper/2311.03969/) (2023)
- [Hypothesis Engineering for Zero-Shot Hate Speech Detection](https://mlsanity.com/paper/2210.00910/) (2022)
- [Towards Legally Enforceable Hate Speech Detection for Public Forums](https://mlsanity.com/paper/2305.13677/) (2023)
- [Something Just Like TRuST : Toxicity Recognition of Span and Target](https://mlsanity.com/paper/2506.02326/) (2025)
- [On the Challenges of Building Datasets for Hate Speech Detection](https://mlsanity.com/paper/2309.02912/) (2023)
- [Toxicity Is a Property of Communicative Events: A Contextual Stress Measurement Framework for NLP](https://mlsanity.com/paper/2503.16072/) (2025)
- [Tell Me What You Know About Sexism: Expert-LLM Interaction Strategies and Co-Created Definitions for Zero-Shot Sexism Detection](https://mlsanity.com/paper/2504.15392/) (2025)
- [Social Hatred: Efficient Multimodal Detection of Hatemongers](https://mlsanity.com/paper/2506.19603/) (2025)
- [Dialogues of Dissent: Thematic and Rhetorical Dimensions of Hate and Counter-Hate Speech in Social Media Conversations](https://mlsanity.com/paper/2507.20528/) (2025)
- [HatePrototypes: Interpretable and Transferable Representations for Implicit and Explicit Hate Speech Detection](https://mlsanity.com/paper/2511.06391/) (2025)
- [Not All Subjectivity Is the Same! Defining Desiderata for the Evaluation of Subjectivity in NLP](https://mlsanity.com/paper/2603.28351/) (2026)
- [Evaluating Large Language Models for Antisemitic Incident Classification](https://mlsanity.com/paper/2607.04890/) (2026)

## Abstract (arXiv)

Offensive Content Warning: This paper contains offensive language only for providing examples that clarify this research and do not reflect the authors' opinions. Please be aware that these examples are offensive and may cause you distress. The subjectivity of recognizing textithate speech makes it a complex task. This is also reflected by different and incomplete definitions in NLP. We present textithate speech criteria, developed with perspectives from law and social science, with the aim of helping researchers create more precise definitions and annotation guidelines on five aspects: (1) target groups, (2) dominance, (3) perpetrator characteristics, (4) type of negative group reference, and the (5) type of potential consequences/effects. Definitions can be structured so that they cover a more broad or more narrow phenomenon. As such, conscious choices can be made on specifying criteria or leaving them open. We argue that the goal and exact task developers have in mind should determine how the scope of textithate speech is defined. We provide an overview of the properties of English datasets from urlhatespeechdata.com that may help select the most suitable dataset for a specific scenario.