ETHOS: an Online Hate Speech Detection Dataset
Not on the map. Why?
Abstract (arXiv)
Online hate speech is a recent problem in our society that is rising at a steady pace by leveraging the vulnerabilities of the corresponding regimes that characterise most social media platforms. This phenomenon is primarily fostered by offensive comments, either during user interaction or in the form of a posted multimedia context. Nowadays, giant corporations own platforms where millions of users log in every day, and protection from exposure to similar phenomena appears to be necessary in order to comply with the corresponding legislation and maintain a high level of service quality. A robust and reliable system for detecting and preventing the uploading of relevant content will have a significant impact on our digitally interconnected society. Several aspects of our daily lives are undeniably linked to our social profiles, making us vulnerable to abusive behaviours. As a result, the lack of accurate hate speech detection mechanisms would severely degrade the overall user experience, although its erroneous operation would pose many ethical concerns. In this paper, we present 'ETHOS', a textual dataset with two variants: binary and multi-label, based on YouTube and Reddit comments validated using the Figure-Eight crowdsourcing platform. Furthermore, we present the annotation protocol used to create this dataset: an active sampling procedure for balancing our data in relation to the various aspects defined. Our key assumption is that, even gaining a small amount of labelled data from such a time-consuming process, we can guarantee hate speech occurrences in the examined material.
Key facts. Cited by 66 papers in the map (data through 7 Oct 2026). First posted on arXiv 11 Jun 2020.
Builds on
No reference list yet.
Built on by
Citing papers per year: 2022: 9; 2023: 8; 2024: 19; 2025: 21; 2026: 9.
66 papers.
Sort:
By area:
- 2023Promptbreeder: Self-Referential Self-Improvement Via Prompt Evolutioncited by 521
- 2026On-Policy Context Distillation for Language Modelscited by 161
- 2022Ground-Truth Labels Matter: A Deeper Look into Input-Label Demonstrationscited by 116
- 2023Evaluating the Social Impact of Generative AI Systems in Systems and Societycited by 91
- 2023Mitigating Label Biases for In-context Learningcited by 88
- 2023Active Learning Principles for In-Context Learning with Large Language Modelscited by 53
- 2024From Languages to Geographies: Towards Evaluating Cultural Bias in Hate Speech Datasetscited by 23
- 2024Stress-Testing Long-Context Language Models with Lifelong ICL and Task Haystackcited by 19
- 2024Efficient and Accurate Prompt Optimization: the Benefit of Memory in Exemplar-Guided Reflectioncited by 18
- 2025System Prompt Optimization with Meta-Learningcited by 18
- 2024Task Facet Learning: A Structured Approach to Prompt Optimizationcited by 17
- 2022KOLD: Korean Offensive Language Datasetcited by 14
38 more
- 2024Misconfidence-based Demonstration Selection for LLM In-Context Learningcited by 14
- 2024MetaHate: A Dataset for Unifying Efforts on Hate Speech Detectioncited by 14
- 2022Deep Learning for Hate Speech Detection: A Comparative Studycited by 13
- 2024Improving Adversarial Data Collection by Supporting Annotators: Lessons from GAHD, a German Hate Speech Datasetcited by 13
- 2024BanTH: A Multi-label Hate Speech Detection Dataset for Transliterated Banglacited by 10
- 2025GAAPO: Genetic Algorithmic Applied to Prompt Optimizationcited by 10
- 2024Introducing MAPO: Momentum-Aided Gradient Descent Prompt Optimizationcited by 8
- 2022"It's Not Just Hate'': A Multi-Dimensional Perspective on Detecting Harmful Speech Onlinecited by 7
- 2023Gen-Z: Generative Zero-Shot Text Classification with Contextualized Label Descriptionscited by 7
- 2022An Attention Matrix for Every Decision: Faithfulness-based Arbitration Among Multiple Attention-Based Interpretations of Transformers in Text Classificationcited by 6
- 2024Investigating Annotator Bias in Large Language Models for Hate Speech Detectioncited by 6
- 2024Beyond Labels: Aligning Large Language Models with Human-like Reasoningcited by 6
- 2025MAPGD: Multi-Agent Prompt Gradient Descent for Collaborative Prompt Optimizationcited by 6
- 2022CoRAL: a Context-aware Croatian Abusive Language Datasetcited by 4
- 2024Zero-to-Strong Generalization: Eliciting Strong Capabilities of Large Language Models Iteratively without Gold Labelscited by 4
- 2022Hypothesis Engineering for Zero-Shot Hate Speech Detectioncited by 3
- 2023Identity Construction in a Misogynist Incels Forumcited by 3
- 2024Quantifying Stereotypes in Languagecited by 3
- 2024Dynamic Uncertainty Ranking: Enhancing Retrieval-Augmented In-Context Learning for Long-Tail Knowledge in LLMscited by 3
- 2025A Checks-and-Balances Framework for Context-Aware Ethical AI Alignmentcited by 3
- 2025Context Engineering for Trustworthiness: Rescorla Wagner Steering Under Mixed and Inappropriate Contextscited by 3
- 2025Prompts Generalize with Low Data: Non-vacuous Generalization Bounds for Optimizing Prompts with More Informative Priorscited by 3
- 2025CoPE: A Small Language Model for Steerable and Scalable Content Labelingcited by 3
- 2026When Context Returns: Toward Robust Internalization in On-Policy Distillationcited by 3
- 2024Effective In-Context Example Selection through Data Compressioncited by 2
- 2024MHS-STMA: Multimodal Hate Speech Detection via Scalable Transformer-Based Multilevel Attention Frameworkcited by 2
- 2024A Survey on Automatic Online Hate Speech Detection in Low-Resource Languagescited by 2
- 2025A Survey of Machine Learning Models and Datasets for the Multi-label Classification of Textual Hate Speech in Englishcited by 2
- 2025LLM Based Bayesian Optimization for Prompt Searchcited by 2
- 2025OutSafe-Bench: A Benchmark for Multimodal Offensive Content Detection in Large Language Modelscited by 2
- 2022Truthful Meta-Explanations for Local Interpretability of Machine Learning Modelscited by 1
- 2023On Bias and Fairness in NLP: Investigating the Impact of Bias and Debiasing in Language Models on the Fairness of Toxicity Detectioncited by 1
- 2025ChildGuard: A Specialized Dataset for Combatting Child-Targeted Hate Speechcited by 1
- 2025Grammar-Guided Evolutionary Search for Discrete Prompt Optimisationcited by 1
- 2025GreenTEA: Gradient Descent with Topic-modeling and Evolutionary Auto-promptingcited by 1
- 2025UniAPO: Unified Multimodal Automated Prompt Optimizationcited by 1
- 2025Semantic Anchors in In-Context Learning: Why Small LLMs Cannot Flip Their Labelscited by 1
- 2026Bye Bye Perspective API: Lessons for Building and Governing Measurement Infrastructurecited by 1
50 most cited of 66 All 66 (JSON)
- 2026Efficient Task Adaptation in Large Language Models: A Survey of Weight-Based, Prompt-Based, and Embedding-Based Adaptations1 Oct 2026
- 2026From Specialization to Generalization: Instruction-tuned LLMs for Robust Harmful Content Mitigation26 Aug 2026
- 2026Counter with Evidence! A Multi-Agent Memory Efficient Reasoning Framework for Hate Category Informed Counterspeech Generation24 Aug 2026
- 2026From Errors to Rules: Iterative Prompt Optimization for Text Classification15 Jun 2026
- 2026When Context Returns: Toward Robust Internalization in On-Policy Distillation10 Jun 2026
- 2026ThreatCore: A Benchmark for Explicit and Implicit Threat Detection11 May 2026
- 2026Bye Bye Perspective API: Lessons for Building and Governing Measurement Infrastructure28 Apr 2026
- 2026On-Policy Context Distillation for Language Models12 Feb 2026
- 2026xList-Hate: A Checklist-Based Framework for Interpretable and Generalizable Hate Speech Detection5 Feb 2026
- 2025CoPE: A Small Language Model for Steerable and Scalable Content Labeling19 Dec 2025
- 2025Adversarially Robust Detection of Harmful Online Content: A Computational Design Science Approach19 Dec 2025
- 2025BanglaSentNet: An Explainable Hybrid Deep Learning Framework for Multi-Aspect Sentiment Analysis with Cross-Domain Transfer Learning28 Nov 2025
38 more
- 2025Semantic Anchors in In-Context Learning: Why Small LLMs Cannot Flip Their Labels26 Nov 2025
- 2025ELPO: Ensemble Learning Based Prompt Optimization for Large Language Models20 Nov 2025
- 2025OutSafe-Bench: A Benchmark for Multimodal Offensive Content Detection in Large Language Models13 Nov 2025
- 2025Prompts Generalize with Low Data: Non-vacuous Generalization Bounds for Optimizing Prompts with More Informative Priors9 Oct 2025
- 2025LLM Based Bayesian Optimization for Prompt Search5 Oct 2025
- 2025MAPGD: Multi-Agent Prompt Gradient Descent for Collaborative Prompt Optimization14 Sep 2025
- 2025Context Engineering for Trustworthiness: Rescorla Wagner Steering Under Mixed and Inappropriate Contexts2 Sep 2025
- 2025UniAPO: Unified Multimodal Automated Prompt Optimization25 Aug 2025
- 2025Transfer Learning via Lexical Relatedness: A Sarcasm and Hate Speech Case Study22 Aug 2025
- 2025GreenTEA: Gradient Descent with Topic-modeling and Evolutionary Auto-prompting12 Aug 2025
- 2025Can NLP Tackle Hate Speech in the Real World? Stakeholder-Informed Feedback and Survey on Counterspeech6 Aug 2025
- 2025Grammar-Guided Evolutionary Search for Discrete Prompt Optimisation14 Jul 2025
- 2025ChildGuard: A Specialized Dataset for Combatting Child-Targeted Hate Speech21 Jun 2025
- 2025Web(er) of Hate: A Survey on How Hate Speech Is Typed19 Jun 2025
- 2025System Prompt Optimization with Meta-Learning14 May 2025
- 2025A Survey of Machine Learning Models and Datasets for the Multi-label Classification of Textual Hate Speech in English11 Apr 2025
- 2025GAAPO: Genetic Algorithmic Applied to Prompt Optimization9 Apr 2025
- 2025A Checks-and-Balances Framework for Context-Aware Ethical AI Alignment31 Jan 2025
- 2024Hostility Detection in UK Politics: A Dataset on Online Abuse Targeting MPs5 Dec 2024
- 2024A Survey on Automatic Online Hate Speech Detection in Low-Resource Languages28 Nov 2024
- 2024Efficient and Accurate Prompt Optimization: the Benefit of Memory in Exemplar-Guided Reflection12 Nov 2024
- 2024Dynamic Uncertainty Ranking: Enhancing Retrieval-Augmented In-Context Learning for Long-Tail Knowledge in LLMs31 Oct 2024
- 2024Introducing MAPO: Momentum-Aided Gradient Descent Prompt Optimization25 Oct 2024
- 2024BanTH: A Multi-label Hate Speech Detection Dataset for Transliterated Bangla17 Oct 2024
- 2024Zero-to-Strong Generalization: Eliciting Strong Capabilities of Large Language Models Iteratively without Gold Labels19 Sep 2024
- 2024MHS-STMA: Multimodal Hate Speech Detection via Scalable Transformer-Based Multilevel Attention Framework8 Sep 2024
- 2024Beyond Labels: Aligning Large Language Models with Human-like Reasoning20 Aug 2024
- 2024Stress-Testing Long-Context Language Models with Lifelong ICL and Task Haystack23 Jul 2024
- 2024Investigating Annotator Bias in Large Language Models for Hate Speech Detection17 Jun 2024
- 2024Task Facet Learning: A Structured Approach to Prompt Optimization15 Jun 2024
- 2024HateDebias: On the Diversity and Variability of Hate Speech Debiasing7 Jun 2024
- 2024Effective In-Context Example Selection through Data Compression19 May 2024
- 2024From Languages to Geographies: Towards Evaluating Cultural Bias in Hate Speech Datasets27 Apr 2024
- 2024Improving Adversarial Data Collection by Supporting Annotators: Lessons from GAHD, a German Hate Speech Dataset28 Mar 2024
- 2024Quantifying Stereotypes in Language28 Jan 2024
- 2024MetaHate: A Dataset for Unifying Efforts on Hate Speech Detection12 Jan 2024
- 2024Misconfidence-based Demonstration Selection for LLM In-Context Learning12 Jan 2024
- 2023Gen-Z: Generative Zero-Shot Text Classification with Contextualized Label Descriptions13 Nov 2023
50 newest of 66 All 66 (JSON)
- 2026On-Policy Context Distillation for Language Modelscited by 161
- 2022Ground-Truth Labels Matter: A Deeper Look into Input-Label Demonstrationscited by 116
- 2023Evaluating the Social Impact of Generative AI Systems in Systems and Societycited by 91
- 2023Mitigating Label Biases for In-context Learningcited by 88
- 2023Active Learning Principles for In-Context Learning with Large Language Modelscited by 53
- 2024From Languages to Geographies: Towards Evaluating Cultural Bias in Hate Speech Datasetscited by 23
- 2024Stress-Testing Long-Context Language Models with Lifelong ICL and Task Haystackcited by 19
- 2024Efficient and Accurate Prompt Optimization: the Benefit of Memory in Exemplar-Guided Reflectioncited by 18
- 2025System Prompt Optimization with Meta-Learningcited by 18
- 2024Task Facet Learning: A Structured Approach to Prompt Optimizationcited by 17
- 2022KOLD: Korean Offensive Language Datasetcited by 14
- 2024Misconfidence-based Demonstration Selection for LLM In-Context Learningcited by 14
12 most cited of 58 All 58 (JSON)
- 2026Efficient Task Adaptation in Large Language Models: A Survey of Weight-Based, Prompt-Based, and Embedding-Based Adaptations1 Oct 2026
- 2026Counter with Evidence! A Multi-Agent Memory Efficient Reasoning Framework for Hate Category Informed Counterspeech Generation24 Aug 2026
- 2026From Errors to Rules: Iterative Prompt Optimization for Text Classification15 Jun 2026
- 2026When Context Returns: Toward Robust Internalization in On-Policy Distillation10 Jun 2026
- 2026ThreatCore: A Benchmark for Explicit and Implicit Threat Detection11 May 2026
- 2026Bye Bye Perspective API: Lessons for Building and Governing Measurement Infrastructure28 Apr 2026
- 2026On-Policy Context Distillation for Language Models12 Feb 2026
- 2026xList-Hate: A Checklist-Based Framework for Interpretable and Generalizable Hate Speech Detection5 Feb 2026
- 2025CoPE: A Small Language Model for Steerable and Scalable Content Labeling19 Dec 2025
- 2025BanglaSentNet: An Explainable Hybrid Deep Learning Framework for Multi-Aspect Sentiment Analysis with Cross-Domain Transfer Learning28 Nov 2025
- 2025Semantic Anchors in In-Context Learning: Why Small LLMs Cannot Flip Their Labels26 Nov 2025
- 2025ELPO: Ensemble Learning Based Prompt Optimization for Large Language Models20 Nov 2025
12 newest of 58 All 58 (JSON)