mlsanityData to 7 Oct 2026

Towards generalisable hate speech detection: a review on obstacles and solutions

arXiv 2102.08886 ↗First posted 17 Feb 2021cs.CLcited by 66

Not on the map. Why?

Abstract (arXiv)

Hate speech is one type of harmful online content which directly attacks or promotes hate towards a group or an individual member based on their actual or perceived aspects of identity, such as ethnicity, religion, and sexual orientation. With online hate speech on the rise, its automatic detection as a natural language processing task is gaining increasing interest. However, it is only recently that it has been shown that existing models generalise poorly to unseen data. This survey paper attempts to summarise how generalisable existing hate speech detection models are, reason why hate speech models struggle to generalise, sums up existing attempts at addressing the main obstacles, and then proposes directions of future research to improve generalisation in hate speech detection.

Key facts. Cited by 66 papers in the map (data through 7 Oct 2026). Builds on 57 of its 143 references that are in the map. First posted on arXiv 17 Feb 2021.

Builds on

57 of 143 references.

References · 57

  1. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,204
  2. 2019RoBERTa: A Robustly Optimized BERT Pretraining Approachcited by 13,266
  3. 2019Unsupervised Cross-lingual Representation Learning at Scalecited by 3,907
  4. 2013Distributed Representations of Words and Phrases and their Compositionalitycited by 3,462
  5. 2016Enriching Word Vectors with Subword Informationcited by 1,578
  6. 2016Man is to Computer Programmer as Woman is to Homemaker? Debiasing Word Embeddingscited by 1,304
  7. 2016Bag of Tricks for Efficient Text Classificationcited by 869
  8. 2017Automated Hate Speech Detection and the Problem of Offensive Languagecited by 724
  9. 2017Word Translation Without Parallel Datacited by 480
  10. 2019Social Bias Frames: Reasoning about Social and Power Implications of Languagecited by 377
  11. 2019Predicting the Type and Target of Offensive Posts in Social Mediacited by 268
  12. 2018Large Scale Crowdsourcing and Characterization of Twitter Abusive Behaviorcited by 265
38 more
  1. 2018Examining Gender and Race Bias in Two Hundred Sentiment Analysis Systemscited by 234
  2. 2016Ex Machina: Personal Attacks Seen at Scalecited by 208
  3. 2019Racial Bias in Hate Speech and Abusive Language Detection Datasetscited by 208
  4. 2017Advances in Pre-Training Distributed Word Representationscited by 207
  5. 2019SemEval-2019 Task 6: Identifying and Categorizing Offensive Language in Social Media (OffensEval)cited by 206
  6. 2020HateBERT: Retraining BERT for Abusive Language Detection in Englishcited by 206
  7. 2018Hate Speech Dataset from a White Supremacy Forumcited by 196
  8. 2016Demographic Dialectal Variation in Social Media: A Case Study of African-American Englishcited by 194
  9. 2017Deep Learning for Hate Speech Detection in Tweetscited by 180
  10. 2018Reducing Gender Bias in Abusive Language Detectioncited by 152
  11. 2020SemEval-2020 Task 12: Multilingual Offensive Language Identification in Social Media (OffensEval 2020)cited by 138
  12. 2017Understanding Abuse: A Typology of Abusive Language Detection Subtaskscited by 115
  13. 2019CONAN -- COunter NArratives through Nichesourcing: a Multilingual Dataset of Responses to Fight Online Hate Speechcited by 109
  14. 2019A BERT-Based Transfer Learning Approach for Hate Speech Detection in Online Social Mediacited by 95
  15. 2020Hate Speech Detection and Racial Bias Mitigation in Social Media based on BERT modelcited by 87
  16. 2020Contextualizing Hate Speech Classifiers with Post-hoc Explanationcited by 84
  17. 2017Detecting Online Hate Speech Using Context Aware Modelscited by 72
  18. 2017Racial Disparity in Natural Language Processing: A Case Study of Social Media African-American Englishcited by 69
  19. 2020Demoting Racial Bias in Hate Speech Detectioncited by 62
  20. 2018All You Need is "Love": Evading Hate-speech Detectioncited by 56
  21. 2018Challenges for Toxic Comment Classification: An In-Depth Error Analysiscited by 55
  22. 2017BPEmb: Tokenization-free Pre-trained Subword Embeddings in 275 Languagescited by 52
  23. 2020Multilingual Twitter Corpus and Baselines for Evaluating Demographic Bias in Hate Speech Recognitioncited by 43
  24. 2018Aggression-annotated Corpus of Hindi-English Code-mixed Datacited by 40
  25. 2020Stereotypical Bias Removal for Hate Speech Detection Task using Knowledge-based Generalizationscited by 40
  26. 2020Detecting East Asian Prejudice on Social Mediacited by 40
  27. 2018Deep Learning for Detecting Cyberbullying Across Multiple Social Media Platformscited by 37
  28. 2018Hate Speech Detection: A Solved Problem? The Challenging Case of Long Tail on Twittercited by 36
  29. 2019Tackling Online Abuse: A Survey of Automated Abuse Detection Methodscited by 33
  30. 2017Mimicking Word Embeddings using Subword RNNscited by 30
  31. 2018Leveraging Intra-User and Inter-User Representation Learning for Automated Hate Speech Detectioncited by 25
  32. 2020Intersectional Bias in Hate Speech and Abusive Language Datasetscited by 22
  33. 2020On Cross-Dataset Generalization in Automatic Detection of Online Abusecited by 20
  34. 2020Detecting White Supremacist Hate Speech using Domain Specific Word Embedding with Deep Learning and BERTcited by 19
  35. 2017Recognizing Explicit and Implicit Hate Speech Using a Weakly Supervised Two-path Bootstrapping Approachcited by 17
  36. 2018Detecting weak and strong Islamophobic hate speech on social mediacited by 15
  37. 2020Treebanking User-Generated Content: a UD Based Overview of Guidelines, Corpora and Unified Recommendationscited by 13
  38. 2018Neural Character-based Composition Models for Abuse Detectioncited by 11

50 most cited of 57 All 57 (JSON)

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2021: 6; 2022: 21; 2023: 20; 2024: 4; 2025: 11; 2026: 4.

66 papers.

Sort:

By area:

  1. 2022A Holistic Approach to Undesired Content Detection in the Real Worldcited by 337
  2. 2022A New Generation of Perspective API: Efficient Multilingual Character-level Transformerscited by 181
  3. 2023Having Beer after Prayer? Measuring Cultural Bias in Large Language Modelscited by 153
  4. 2022Pile of Law: Learning Responsible Data Filtering from the Law and a 256GB Open-Source Legal Datasetcited by 113
  5. 2022Handling Bias in Toxic Speech Detection: A Surveycited by 60
  6. 2023When Do Annotator Demographics Matter? Measuring the Influence of Annotator Demographics with the POPQUORN Datasetcited by 51
  7. 2023Assessing Language Model Deployment with Risk Cardscited by 30
  8. 2023Interpretable Unified Language Checkingcited by 26
  9. 2022How Hate Speech Varies by Target Identity: A Computational Analysiscited by 23
  10. 2021Overview of the HASOC Subtrack at FIRE 2021: Hate Speech and Offensive Content Identification in English and Indo-Aryan Languagescited by 17
  11. 2023Understanding Counterspeech for Online Harm Mitigationcited by 17
  12. 2022Improving Generalizability in Implicitly Abusive Language Detection with Concept Activation Vectorscited by 14
38 more
  1. 2022Deep Learning for Hate Speech Detection: A Comparative Studycited by 13
  2. 2022AnnoBERT: Effectively Representing Multiple Annotators' Label Choices to Improve Hate Speech Detectioncited by 12
  3. 2025Lost in Moderation: How Commercial Content Moderation APIs Over- and Under-Moderate Group-Targeted Hate Speech and Linguistic Variationscited by 12
  4. 2022Session-based Cyberbullying Detection in Social Media: A Surveycited by 10
  5. 2022Generalizing Hate Speech Detection Using Multi-Task Learning: A Case Study of Political Public Figurescited by 10
  6. 2022Multilingual Auxiliary Tasks Training: Bridging the Gap between Languages for Zero-Shot Transfer of Hate Speech Detection Modelscited by 10
  7. 2022Detecting Harmful Online Conversational Content towards LGBTQIA+ Individualscited by 8
  8. 2025Enhanced Multimodal Hate Video Detection via Channel-wise and Modality-wise Fusioncited by 8
  9. 2021Offensive Language Identification in Low-resourced Code-mixed Dravidian languages using Pseudo-labelingcited by 6
  10. 2022Cyberbullying detection across social media platforms via platform-aware adversarial encodingcited by 6
  11. 2022Hidden behind the obvious: misleading keywords and implicitly abusive language on social mediacited by 6
  12. 2023Antisemitic Messages? A Guide to High-Quality Annotation and a Labeled Dataset of Tweetscited by 6
  13. 2023CFL: Causally Fair Language Models Through Token-level Attribute Controlled Generationcited by 6
  14. 2023PEACE: Cross-Platform Hate Speech Detection- A Causality-guided Frameworkcited by 6
  15. 2023BAN-PL: a Novel Polish Dataset of Banned Harmful and Offensive Content from Wykop.pl web servicecited by 6
  16. 2021Cross-lingual Capsule Network for Hate Speech Detection in Social Mediacited by 5
  17. 2021Sexism Prediction in Spanish and English Tweets Using Monolingual and Multilingual BERT and Ensemble Modelscited by 5
  18. 2021"Stop Asian Hate!" : Refining Detection of Anti-Asian Hate Speech During the COVID-19 Pandemiccited by 4
  19. 2023Towards Legally Enforceable Hate Speech Detection for Public Forumscited by 4
  20. 2025Mario at EXIST 2025: A Simple Gateway to Effective Multilingual Sexism Detectioncited by 4
  21. 2022Dynamically Refined Regularization for Improving Cross-corpora Hate Speech Detectioncited by 3
  22. 2022Hypothesis Engineering for Zero-Shot Hate Speech Detectioncited by 3
  23. 2022Human-in-the-Loop Hate Speech Classification in a Multilingual Contextcited by 3
  24. 2023Concept-Based Explanations to Test for False Causal Relationships Learned by Abusive Language Classifierscited by 3
  25. 2024GPT-HateCheck: Can LLMs Write Better Functional Tests for Hate Speech Detection?cited by 3
  26. 2023Towards Weakly-Supervised Hate Speech Classification Across Datasetscited by 2
  27. 2023Examining the Limitations of Computational Rumor Detection Models Trained on Static Datasetscited by 2
  28. 2023Focal Inferential Infusion Coupled with Tractable Density Discrimination for Implicit Hate Detectioncited by 2
  29. 2023Towards a Unified Framework for Adaptable Problematic Content Detection via Continual Learningcited by 2
  30. 2024Cross-lingual Offensive Language Detection: A Systematic Review of Datasets, Transfer Approaches and Challengescited by 2
  31. 2025A Survey of Machine Learning Models and Datasets for the Multi-label Classification of Textual Hate Speech in Englishcited by 2
  32. 2025System Report for CCL25-Eval Task 10: SRAG-MAV for Fine-Grained Chinese Hate Speech Recognitioncited by 2
  33. 2025Aetheria: A multimodal interpretable content safety framework based on multi-agent debate and collaborationcited by 2
  34. 2023Evaluating the Effectiveness of Natural Language Inference for Hate Speech Detection in Languages with Limited Labeled Datacited by 1
  35. 2023Are You Worthy of My Trust?: A Socioethical Perspective on the Impacts of Trustworthy AI Systems on the Environment and Human Societycited by 1
  36. 2023Meta learning with language models: Challenges and opportunities in the classification of imbalanced textcited by 1
  37. 2025MultiHateLoc: Towards Temporal Localisation of Multimodal Hate Content in Online Videoscited by 1
  38. 2021Weakly Supervised Cross-platform Teenager Detection with Adversarial BERTcited by 0

50 most cited of 66 All 66 (JSON)