# NaijaHate: Evaluating Hate Speech Detection on Nigerian Twitter Using Representative Data arXiv: https://arxiv.org/abs/2403.19260 ยท HTML: https://mlsanity.com/paper/2403.19260/ ## Key facts - Cited by 10 papers in the map (data through 14 Oct 2026). - Builds on 27 of its 76 references that are in the map. - First posted on arXiv 28 Mar 2024. ## Builds on (most cited first, 27 of 27) - [Unsupervised Cross-lingual Representation Learning at Scale](https://mlsanity.com/paper/1911.02116/) (2019) - [Don't Stop Pretraining: Adapt Language Models to Domains and Tasks](https://mlsanity.com/paper/2004.10964/) (2020) - [DeBERTaV3: Improving DeBERTa using ELECTRA-Style Pre-Training with Gradient-Disentangled Embedding Sharing](https://mlsanity.com/paper/2111.09543/) (2021) - [Automated Hate Speech Detection and the Problem of Offensive Language](https://mlsanity.com/paper/1703.04009/) (2017) - [Towards General Text Embeddings with Multi-stage Contrastive Learning](https://mlsanity.com/paper/2308.03281/) (2023) - [HateXplain: A Benchmark Dataset for Explainable Hate Speech Detection](https://mlsanity.com/paper/2012.10289/) (2020) - [A Holistic Approach to Undesired Content Detection in the Real World](https://mlsanity.com/paper/2208.03274/) (2022) - [BERTweet: A pre-trained language model for English Tweets](https://mlsanity.com/paper/2005.10200/) (2020) - [Predicting the Type and Target of Offensive Posts in Social Media](https://mlsanity.com/paper/1902.09666/) (2019) - [HateCheck: Functional Tests for Hate Speech Detection Models](https://mlsanity.com/paper/2012.15606/) (2020) - [HateBERT: Retraining BERT for Abusive Language Detection in English](https://mlsanity.com/paper/2010.12472/) (2020) - [A New Generation of Perspective API: Efficient Multilingual Character-level Transformers](https://mlsanity.com/paper/2202.11176/) (2022) - [MasakhaNER: Named Entity Recognition for African Languages](https://mlsanity.com/paper/2103.11811/) (2021) - [Adapting Pre-trained Language Models to African Languages via Multilingual Adaptive Fine-Tuning](https://mlsanity.com/paper/2204.06487/) (2022) - [Lifting the Curse of Multilinguality by Pre-training Modular Transformers](https://mlsanity.com/paper/2205.06266/) (2022) - [MultiVerS: Improving scientific claim verification with weak supervision and full-document context](https://mlsanity.com/paper/2112.01640/) (2021) - [XLM-T: Multilingual Language Models in Twitter for Sentiment Analysis and Beyond](https://mlsanity.com/paper/2104.12250/) (2021) - [Human-AI Collaboration via Conditional Delegation: A Case Study of Content Moderation](https://mlsanity.com/paper/2204.11788/) (2022) - [Detecting East Asian Prejudice on Social Media](https://mlsanity.com/paper/2005.03909/) (2020) - [Detecting Cross-Geographic Biases in Toxicity Modeling on Social Media](https://mlsanity.com/paper/2104.06999/) (2021) - [FBERT: A Neural Transformer for Identifying Offensive Content](https://mlsanity.com/paper/2109.05074/) (2021) - [On Cross-Dataset Generalization in Automatic Detection of Online Abuse](https://mlsanity.com/paper/2010.07414/) (2020) - [HERDPhobia: A Dataset for Hate Speech against Fulani in Nigeria](https://mlsanity.com/paper/2211.15262/) (2022) - [Multilingual Detection of Personal Employment Status on Twitter](https://mlsanity.com/paper/2203.09178/) (2022) - [Tackling Hate Speech in Low-resource Languages with Context Experts](https://mlsanity.com/paper/2303.16828/) (2023) - [Social Approval and Network Homophily as Motivators of Online Toxicity](https://mlsanity.com/paper/2310.07779/) (2023) - [Detection and Analysis of Offensive Online Content in Hausa Language](https://mlsanity.com/paper/2311.10541/) (2023) ## Built on by (most cited first, 10 of 10) - [From Languages to Geographies: Towards Evaluating Cultural Bias in Hate Speech Datasets](https://mlsanity.com/paper/2404.17874/) (2024) - [HateDay: Insights from a Global Hate Speech Dataset Representative of a Day on Twitter](https://mlsanity.com/paper/2411.15462/) (2024) - [AfriHate: A Multilingual Collection of Hate Speech and Abusive Language Datasets for African Languages](https://mlsanity.com/paper/2501.08284/) (2025) - [On the Risk of Evidence Pollution for Malicious Social Text Detection in the Era of LLMs](https://mlsanity.com/paper/2410.12600/) (2024) - [When Claims Evolve: Evaluating and Enhancing the Robustness of Embedding Models Against Misinformation Edits](https://mlsanity.com/paper/2503.03417/) (2025) - [Culture Cartography: Mapping the Landscape of Cultural Knowledge](https://mlsanity.com/paper/2510.27672/) (2025) - [A Multi-Task Benchmark for Abusive Language Detection in Low-Resource Settings](https://mlsanity.com/paper/2505.12116/) (2025) - [A multilingual dataset for offensive language and hate speech detection for hausa, yoruba and igbo languages](https://mlsanity.com/paper/2406.02169/) (2024) - [NaijaNLP: A Survey of Nigerian Low-Resource Languages](https://mlsanity.com/paper/2502.19784/) (2025) - [GENDIS: Gender-Related Discourse and Stance in Multilingual YouTube Comments from India, Kenya, and Nigeria](https://mlsanity.com/paper/2610.17778/) (2026) ## Abstract (arXiv) To address the global issue of online hate, hate speech detection (HSD) systems are typically developed on datasets from the United States, thereby failing to generalize to English dialects from the Majority World. Furthermore, HSD models are often evaluated on non-representative samples, raising concerns about overestimating model performance in real-world settings. In this work, we introduce NaijaHate, the first dataset annotated for HSD which contains a representative sample of Nigerian tweets. We demonstrate that HSD evaluated on biased datasets traditionally used in the literature consistently overestimates real-world performance by at least two-fold. We then propose NaijaXLM-T, a pretrained model tailored to the Nigerian Twitter context, and establish the key role played by domain-adaptive pretraining and finetuning in maximizing HSD performance. Finally, owing to the modest performance of HSD systems in real-world conditions, we find that content moderators would need to review about ten thousand Nigerian tweets flagged as hateful daily to moderate 60% of all hateful content, highlighting the challenges of moderating hate speech at scale as social media usage continues to grow globally. Taken together, these results pave the way towards robust HSD systems and a better protection of social media users from hateful content in low-resource settings.