# Robust Hate Speech Detection in Social Media: A Cross-Dataset Empirical Evaluation

arXiv: https://arxiv.org/abs/2307.01680 · HTML: https://mlsanity.com/paper/2307.01680/

## Key facts

- Cited by 28 papers in the map (data through 14 Oct 2026).
- Builds on 21 of its 52 references that are in the map.
- First posted on arXiv 4 Jul 2023.

## Builds on (most cited first, 21 of 21)

- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [RoBERTa: A Robustly Optimized BERT Pretraining Approach](https://mlsanity.com/paper/1907.11692/) (2019)
- [Enriching Word Vectors with Subword Information](https://mlsanity.com/paper/1607.04606/) (2016)
- [Automated Hate Speech Detection and the Problem of Offensive Language](https://mlsanity.com/paper/1703.04009/) (2017)
- [HateXplain: A Benchmark Dataset for Explainable Hate Speech Detection](https://mlsanity.com/paper/2012.10289/) (2020)
- [Tune: A Research Platform for Distributed Model Selection and Training](https://mlsanity.com/paper/1807.05118/) (2018)
- [BERTweet: A pre-trained language model for English Tweets](https://mlsanity.com/paper/2005.10200/) (2020)
- [Predicting the Type and Target of Offensive Posts in Social Media](https://mlsanity.com/paper/1902.09666/) (2019)
- [Large Scale Crowdsourcing and Characterization of Twitter Abusive Behavior](https://mlsanity.com/paper/1802.00393/) (2018)
- [SemEval-2020 Task 12: Multilingual Offensive Language Identification in Social Media (OffensEval 2020)](https://mlsanity.com/paper/2006.07235/) (2020)
- [Multilingual and Multi-Aspect Hate Speech Analysis](https://mlsanity.com/paper/1908.11049/) (2019)
- [TimeLMs: Diachronic Language Models from Twitter](https://mlsanity.com/paper/2202.03829/) (2022)
- [TweetNLP: Cutting-Edge Natural Language Processing for Social Media](https://mlsanity.com/paper/2206.14774/) (2022)
- [Measuring a hate speech spectrum with faceted Rasch item response theory and perspective-aware, explainable-by-design deep learning](https://mlsanity.com/paper/2009.10277/) (2020)
- ["Call me sexist, but...": Revisiting Sexism Detection Using Psychological Scales and Adversarial Samples](https://mlsanity.com/paper/2004.12764/) (2020)
- [Hate Towards the Political Opponent: A Twitter Corpus Study of the 2020 US Elections on the Basis of Offensive Speech and Stance Detection](https://mlsanity.com/paper/2103.01664/) (2021)
- [Detecting East Asian Prejudice on Social Media](https://mlsanity.com/paper/2005.03909/) (2020)
- [Overview of the HASOC track at FIRE 2020: Hate Speech and Offensive Content Identification in Indo-European Languages](https://mlsanity.com/paper/2108.05927/) (2021)
- [Twitter Topic Classification](https://mlsanity.com/paper/2209.09824/) (2022)
- [Understanding the temporal evolution of COVID-19 research through machine learning and natural language processing](https://mlsanity.com/paper/2007.11604/) (2020)
- [Dynamically Refined Regularization for Improving Cross-corpora Hate Speech Detection](https://mlsanity.com/paper/2203.12536/) (2022)

## Built on by (most cited first, 28 of 28)

- [ToxiCraft: A Novel Framework for Synthetic Generation of Harmful Information](https://mlsanity.com/paper/2409.14740/) (2024)
- [HateBench: Benchmarking Hate Speech Detectors on LLM-Generated Content and Hate Campaigns](https://mlsanity.com/paper/2501.16750/) (2025)
- [Causality Guided Disentanglement for Cross-Platform Hate Speech Detection](https://mlsanity.com/paper/2308.02080/) (2023)
- [HateModerate: Testing Hate Speech Detectors against Content Moderation Policies](https://mlsanity.com/paper/2307.12418/) (2023)
- [Generative AI for Hate Speech Detection: Evaluation and Findings](https://mlsanity.com/paper/2311.09993/) (2023)
- [DefVerify: Do Hate Speech Models Reflect Their Dataset's Definition?](https://mlsanity.com/paper/2410.15911/) (2024)
- [LLMs and Finetuning: Benchmarking cross-domain performance for hate speech detection](https://mlsanity.com/paper/2310.18964/) (2023)
- [Probing Critical Learning Dynamics of PLMs for Hate Speech Detection](https://mlsanity.com/paper/2402.02144/) (2024)
- [Evaluating Large Language Models for Detecting Antisemitism](https://mlsanity.com/paper/2509.18293/) (2025)
- [Toxicity in Online Platforms and AI Systems: A Survey of Needs, Challenges, Mitigations, and Future Directions](https://mlsanity.com/paper/2509.25539/) (2025)
- [Data-Efficient Hate Speech Detection via Cross-Lingual Nearest Neighbor Retrieval with Limited Labeled Data](https://mlsanity.com/paper/2505.14272/) (2025)
- [Are you sure? Measuring models bias in content moderation through uncertainty](https://mlsanity.com/paper/2509.22699/) (2025)
- [Measuring Online Hate on 4chan using Pre-trained Deep Learning Models](https://mlsanity.com/paper/2504.00045/) (2025)
- [Relation Extraction or Pattern Matching? Unravelling the Generalisation Limits of Language Models for Biographical RE](https://mlsanity.com/paper/2505.12533/) (2025)
- [DoDo Learning: DOmain-DemOgraphic Transfer in Language Models for Detecting Abuse Targeted at Public Figures](https://mlsanity.com/paper/2307.16811/) (2023)
- [Causal Effects of Trigger Words in Social Media Discussions: A Large-Scale Case Study about UK Politics on Reddit](https://mlsanity.com/paper/2405.10213/) (2024)
- [The Unappreciated Role of Intent in Algorithmic Moderation of Social Media Content](https://mlsanity.com/paper/2405.11030/) (2024)
- [Hostility Detection in UK Politics: A Dataset on Online Abuse Targeting MPs](https://mlsanity.com/paper/2412.04046/) (2024)
- [Detoxify: A framework for abusive text transformation using LLMs](https://mlsanity.com/paper/2507.10177/) (2025)
- [SMARTER: A Data-efficient Framework to Improve Toxicity Detection with Explanation via Self-augmenting Large Language Models](https://mlsanity.com/paper/2509.15174/) (2025)
- [Muse-it: A Tool for Analyzing Music Discourse on Reddit](https://mlsanity.com/paper/2509.20228/) (2025)
- [A Machine Learning Approach for Detection of Mental Health Conditions and Cyberbullying from Social Media](https://mlsanity.com/paper/2511.20001/) (2025)
- [Bias Dynamics in BabyLMs: Towards a Compute-Efficient Sandbox for Democratising Pre-Training Debiasing](https://mlsanity.com/paper/2601.09421/) (2026)
- [Do We Still Need Humans in the Loop? Human vs. LLM Annotation in Active Learning for TikTok Hate Speech Detection](https://mlsanity.com/paper/2604.13899/) (2026)
- [A Multi-Dimensional Audit of Politically Aligned Large Language Models](https://mlsanity.com/paper/2604.24429/) (2026)
- [Who Decides What Is Harmful? Content Moderation Policy Through A Multi-Agent Personalised Inference Framework](https://mlsanity.com/paper/2605.01416/) (2026)
- [From Specialization to Generalization: Instruction-tuned LLMs for Robust Harmful Content Mitigation](https://mlsanity.com/paper/2608.25605/) (2026)
- [SDARE-Bench: Evaluating Large Language Models on Conversational Stigma Detection and Response in Dyadic and Group Dialogue](https://mlsanity.com/paper/2609.01548/) (2026)

## Abstract (arXiv)

The automatic detection of hate speech online is an active research area in NLP. Most of the studies to date are based on social media datasets that contribute to the creation of hate speech detection models trained on them. However, data creation processes contain their own biases, and models inherently learn from these dataset-specific biases. In this paper, we perform a large-scale cross-dataset comparison where we fine-tune language models on different hate speech detection datasets. This analysis shows how some datasets are more generalisable than others when used as training data. Crucially, our experiments show how combining hate speech detection datasets can contribute to the development of robust hate speech detection models. This robustness holds even when controlling by data size and compared with the best individual datasets.