# HABERTOR: An Efficient and Effective Deep Hatespeech Detector

arXiv: https://arxiv.org/abs/2010.08865 · HTML: https://mlsanity.com/paper/2010.08865/

## Key facts

- Cited by 14 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 17 Oct 2020.

## Built on by (most cited first, 14 of 14)

- [Temporal Adaptation of BERT and Performance on Downstream Document Classification: Insights from Social Media](https://mlsanity.com/paper/2104.08116/) (2021)
- [Character-level HyperNetworks for Hate Speech Detection](https://mlsanity.com/paper/2111.06336/) (2021)
- ["Define Your Terms" : Enhancing Efficient Offensive Speech Classification with Definition](https://mlsanity.com/paper/2402.03221/) (2024)
- [ToXCL: A Unified Framework for Toxic Speech Detection and Explanation](https://mlsanity.com/paper/2403.16685/) (2024)
- [BERT-Beta: A Proactive Probabilistic Approach to Text Moderation](https://mlsanity.com/paper/2109.08805/) (2021)
- [Hate Speech and Offensive Content Detection in Indo-Aryan Languages: A Battle of LSTM and Transformers](https://mlsanity.com/paper/2312.05671/) (2023)
- [Leveraging Transformers for Hate Speech Detection in Conversational Code-Mixed Tweets](https://mlsanity.com/paper/2112.09986/) (2021)
- [Not All Comments are Equal: Insights into Comment Moderation from a Topic-Aware Model](https://mlsanity.com/paper/2109.10033/) (2021)
- [Deep Learning Approaches for Detecting Adversarial Cyberbullying and Hate Speech in Social Networks](https://mlsanity.com/paper/2406.17793/) (2024)
- [Neural Networks at a Fraction with Pruned Quaternions](https://mlsanity.com/paper/2308.06780/) (2023)
- [Reverse Constitutional AI: A Framework for Controllable Toxic Data Generation via Probability-Clamped RLAIF](https://mlsanity.com/paper/2604.17769/) (2026)
- [An Effective, Robust and Fairness-aware Hate Speech Detection Framework](https://mlsanity.com/paper/2409.17191/) (2024)
- [U-GIFT: Uncertainty-Guided Firewall for Toxic Speech in Few-Shot Scenario](https://mlsanity.com/paper/2501.00907/) (2025)
- [RanAT4BIE: Random Adversarial Training for Biomedical Information Extraction](https://mlsanity.com/paper/2509.11191/) (2025)

## Abstract (arXiv)

We present our HABERTOR model for detecting hatespeech in large scale user-generated content. Inspired by the recent success of the BERT model, we propose several modifications to BERT to enhance the performance on the downstream hatespeech classification task. HABERTOR inherits BERT's architecture, but is different in four aspects: (i) it generates its own vocabularies and is pre-trained from the scratch using the largest scale hatespeech dataset; (ii) it consists of Quaternion-based factorized components, resulting in a much smaller number of parameters, faster training and inferencing, as well as less memory usage; (iii) it uses our proposed multi-source ensemble heads with a pooling layer for separate input sources, to further enhance its effectiveness; and (iv) it uses a regularized adversarial training with our proposed fine-grained and adaptive noise magnitude to enhance its robustness. Through experiments on the large-scale real-world hatespeech dataset with 1.4M annotated comments, we show that HABERTOR works better than 15 state-of-the-art hatespeech detection methods, including fine-tuning Language Models. In particular, comparing with BERT, our HABERTOR is 4~5 times faster in the training/inferencing phase, uses less than 1/3 of the memory, and has better performance, even though we pre-train it by using less than 1% of the number of words. Our generalizability analysis shows that HABERTOR transfers well to other unseen hatespeech datasets and is a more efficient and effective alternative to BERT for the hatespeech classification.