# Hate Speech Detection: A Solved Problem? The Challenging Case of Long Tail on Twitter

arXiv: https://arxiv.org/abs/1803.03662 · HTML: https://mlsanity.com/paper/1803.03662/

## Key facts

- Cited by 36 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 27 Feb 2018.

## Built on by (most cited first, 30 of 36)

- [Improving alignment of dialogue agents via targeted human judgements](https://mlsanity.com/paper/2209.14375/) (2022)
- [Towards generalisable hate speech detection: a review on obstacles and solutions](https://mlsanity.com/paper/2102.08886/) (2021)
- [Whose Opinions Matter? Perspective-aware Models to Identify Opinions of Hate Speech Victims in Abusive Language Detection](https://mlsanity.com/paper/2106.15896/) (2021)
- [Hate Speech and Counter Speech Detection: Conversational Context Does Matter](https://mlsanity.com/paper/2206.06423/) (2022)
- [A systematic review of Hate Speech automatic detection using Natural Language Processing](https://mlsanity.com/paper/2106.00742/) (2021)
- [DISARM: Detecting the Victims Targeted by Harmful Memes](https://mlsanity.com/paper/2205.05738/) (2022)
- [KOLD: Korean Offensive Language Dataset](https://mlsanity.com/paper/2205.11315/) (2022)
- [Characterizing the Entities in Harmful Memes: Who is the Hero, the Villain, the Victim?](https://mlsanity.com/paper/2301.11219/) (2023)
- [AOMD: An Analogy-aware Approach to Offensive Meme Detection on Social Media](https://mlsanity.com/paper/2106.11229/) (2021)
- [Generalizing Hate Speech Detection Using Multi-Task Learning: A Case Study of Political Public Figures](https://mlsanity.com/paper/2208.10598/) (2022)
- [One to rule them all: Towards Joint Indic Language Hate Speech Detection](https://mlsanity.com/paper/2109.13711/) (2021)
- [Hidden behind the obvious: misleading keywords and implicitly abusive language on social media](https://mlsanity.com/paper/2205.01374/) (2022)
- [Sexism detection: The first corpus in Algerian dialect with a code-switching in Arabic/ French and English](https://mlsanity.com/paper/2104.01443/) (2021)
- [Leveraging cross-platform data to improve automated hate speech detection](https://mlsanity.com/paper/2102.04895/) (2021)
- [Contextual Hate Speech Detection in Code Mixed Text using Transformer Based Approaches](https://mlsanity.com/paper/2110.09338/) (2021)
- [Hatemongers ride on echo chambers to escalate hate speech diffusion](https://mlsanity.com/paper/2302.02479/) (2023)
- [Mario at EXIST 2025: A Simple Gateway to Effective Multilingual Sexism Detection](https://mlsanity.com/paper/2507.10996/) (2025)
- [Ceasing hate withMoH: Hate Speech Detection in Hindi-English Code-Switched Language](https://mlsanity.com/paper/2110.09393/) (2021)
- [Cross-lingual Hate Speech Detection using Transformer Models](https://mlsanity.com/paper/2111.00981/) (2021)
- [Selecting and combining complementary feature representations and classifiers for hate speech detection](https://mlsanity.com/paper/2201.06721/) (2022)
- [Sexism Identification in Tweets and Gabs using Deep Neural Networks](https://mlsanity.com/paper/2111.03612/) (2021)
- [Combining Textual Features for the Detection of Hateful and Offensive Language](https://mlsanity.com/paper/2112.04803/) (2021)
- [Anti-Asian Hate Speech Detection via Data Augmented Semantic Relation Inference](https://mlsanity.com/paper/2204.07010/) (2022)
- [Addressing Hate Speech with Data Science: An Overview from Computer Science Perspective](https://mlsanity.com/paper/2103.10489/) (2021)
- [Balancing Fairness and Robustness via Partial Invariance](https://mlsanity.com/paper/2112.09346/) (2021)
- [Machines Do See Color: A Guideline to Classify Different Forms of Racist Discourse in Large Corpora](https://mlsanity.com/paper/2401.09333/) (2024)
- [SafeSpeech: A Comprehensive and Interactive Tool for Analysing Sexist and Abusive Language in Conversations](https://mlsanity.com/paper/2503.06534/) (2025)
- [Combating Toxic Language: A Review of LLM-Based Strategies for Software Engineering](https://mlsanity.com/paper/2504.15439/) (2025)
- [YZR-net : Self-supervised Hidden representations Invariant to Transformations for profanity detection](https://mlsanity.com/paper/2211.15532/) (2022)
- [Poster: Link between Bias, Node Sensitivity and Long-Tail Distribution in trained DNNs](https://mlsanity.com/paper/2303.16589/) (2023)

## Abstract (arXiv)

In recent years, the increasing propagation of hate speech on social media and the urgent need for effective counter-measures have drawn significant investment from governments, companies, and researchers. A large number of methods have been developed for automated hate speech detection online. This aims to classify textual content into non-hate or hate speech, in which case the method may also identify the targeting characteristics (i.e., types of hate, such as race, and religion) in the hate speech. However, we notice significant difference between the performance of the two (i.e., non-hate v.s. hate). In this work, we argue for a focus on the latter problem for practical reasons. We show that it is a much more challenging task, as our analysis of the language in the typical datasets shows that hate speech lacks unique, discriminative features and therefore is found in the 'long tail' in a dataset that is difficult to discover. We then propose Deep Neural Network structures serving as feature extractors that are particularly effective for capturing the semantics of hate speech. Our methods are evaluated on the largest collection of hate speech datasets based on Twitter, and are shown to be able to outperform the best performing method by up to 5 percentage points in macro-average F1, or 8 percentage points in the more challenging case of identifying hateful content.