# On Cross-Dataset Generalization in Automatic Detection of Online Abuse

arXiv: https://arxiv.org/abs/2010.07414 · HTML: https://mlsanity.com/paper/2010.07414/

## Key facts

- Cited by 20 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 14 Oct 2020.

## Built on by (most cited first, 20 of 20)

- [Auditing large language models: a three-layered approach](https://mlsanity.com/paper/2302.08500/) (2023)
- [State-of-the-art generalisation research in NLP: A taxonomy and review](https://mlsanity.com/paper/2210.03050/) (2022)
- [Revisiting Out-of-distribution Robustness in NLP: Benchmark, Analysis, and LLMs Evaluations](https://mlsanity.com/paper/2306.04618/) (2023)
- [Towards generalisable hate speech detection: a review on obstacles and solutions](https://mlsanity.com/paper/2102.08886/) (2021)
- [ConvAbuse: Data, Analysis, and Benchmarks for Nuanced Abuse Detection in Conversational AI](https://mlsanity.com/paper/2109.09483/) (2021)
- [From Languages to Geographies: Towards Evaluating Cultural Bias in Hate Speech Datasets](https://mlsanity.com/paper/2404.17874/) (2024)
- [Necessity and Sufficiency for Explaining Text Classifiers: A Case Study in Hate Speech Detection](https://mlsanity.com/paper/2205.03302/) (2022)
- [Improving Generalizability in Implicitly Abusive Language Detection with Concept Activation Vectors](https://mlsanity.com/paper/2204.02261/) (2022)
- [HateDay: Insights from a Global Hate Speech Dataset Representative of a Day on Twitter](https://mlsanity.com/paper/2411.15462/) (2024)
- [NaijaHate: Evaluating Hate Speech Detection on Nigerian Twitter Using Representative Data](https://mlsanity.com/paper/2403.19260/) (2024)
- [Detecting Harmful Online Conversational Content towards LGBTQIA+ Individuals](https://mlsanity.com/paper/2207.10032/) (2022)
- [Hidden behind the obvious: misleading keywords and implicitly abusive language on social media](https://mlsanity.com/paper/2205.01374/) (2022)
- [Cyberbullying Classifiers are Sensitive to Model-Agnostic Perturbations](https://mlsanity.com/paper/2201.06384/) (2022)
- [KTCR: Improving Implicit Hate Detection with Knowledge Transfer driven Concept Refinement](https://mlsanity.com/paper/2410.15314/) (2024)
- [Latent Feature-based Data Splits to Improve Generalisation Evaluation: A Hate Speech Detection Case Study](https://mlsanity.com/paper/2311.10236/) (2023)
- [Topic Bias in Emotion Classification](https://mlsanity.com/paper/2312.09043/) (2023)
- [Improving Hate Speech Classification with Cross-Taxonomy Dataset Integration](https://mlsanity.com/paper/2503.05357/) (2025)
- [Tackling Social Bias against the Poor: A Dataset and Taxonomy on Aporophobia](https://mlsanity.com/paper/2504.13085/) (2025)
- [Aligning Implied Statements for Implicit Hate Speech Generalizability with Context-Bounded Semi-hard Negative Mining](https://mlsanity.com/paper/2606.18852/) (2026)
- [The Cross-Domain Generalization Cost of Offensive Language Detection](https://mlsanity.com/paper/2607.23512/) (2026)

## Abstract (arXiv)

NLP research has attained high performances in abusive language detection as a supervised classification task. While in research settings, training and test datasets are usually obtained from similar data samples, in practice systems are often applied on data that are different from the training set in topic and class distributions. Also, the ambiguity in class definitions inherited in this task aggravates the discrepancies between source and target datasets. We explore the topic bias and the task formulation bias in cross-dataset generalization. We show that the benign examples in the Wikipedia Detox dataset are biased towards platform-specific topics. We identify these examples using unsupervised topic modeling and manual inspection of topics' keywords. Removing these topics increases cross-dataset generalization, without reducing in-domain classification performance. For a robust dataset design, we suggest applying inexpensive unsupervised methods to inspect the collected data and downsize the non-generalizable content before manually annotating for class labels.