# Detection of Hate Speech using BERT and Hate Speech Word Embedding with Deep Model

arXiv: https://arxiv.org/abs/2111.01515 · HTML: https://mlsanity.com/paper/2111.01515/

## Key facts

- Cited by 26 papers in the map (data through 14 Oct 2026).
- Builds on 20 of its 62 references that are in the map.
- First posted on arXiv 2 Nov 2021.

## Builds on (most cited first, 20 of 20)

- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- ["Why Should I Trust You?": Explaining the Predictions of Any Classifier](https://mlsanity.com/paper/1602.04938/) (2016)
- [GLUE: A Multi-Task Benchmark and Analysis Platform for Natural Language Understanding](https://mlsanity.com/paper/1804.07461/) (2018)
- [Efficient Estimation of Word Representations in Vector Space](https://mlsanity.com/paper/1301.3781/) (2013)
- [SQuAD: 100,000+ Questions for Machine Comprehension of Text](https://mlsanity.com/paper/1606.05250/) (2016)
- [A Broad-Coverage Challenge Corpus for Sentence Understanding through Inference](https://mlsanity.com/paper/1704.05426/) (2017)
- [Deep contextualized word representations](https://mlsanity.com/paper/1802.05365/) (2018)
- [Automated Hate Speech Detection and the Problem of Offensive Language](https://mlsanity.com/paper/1703.04009/) (2017)
- [Large Scale Crowdsourcing and Characterization of Twitter Abusive Behavior](https://mlsanity.com/paper/1802.00393/) (2018)
- [Hate Speech Dataset from a White Supremacy Forum](https://mlsanity.com/paper/1809.04444/) (2018)
- [Deep Learning for Hate Speech Detection in Tweets](https://mlsanity.com/paper/1706.00188/) (2017)
- [Hate Speech Detection and Racial Bias Mitigation in Social Media based on BERT model](https://mlsanity.com/paper/2008.06460/) (2020)
- [Detecting Offensive Language in Tweets Using Deep Learning](https://mlsanity.com/paper/1801.04433/) (2018)
- [Detecting the Hate Code on Social Media](https://mlsanity.com/paper/1703.05443/) (2017)
- [Predicting online extremism, content adopters, and interaction reciprocity](https://mlsanity.com/paper/1605.00659/) (2016)
- [Part-of-Speech Tagging with Bidirectional Long Short-Term Memory Recurrent Neural Network](https://mlsanity.com/paper/1510.06168/) (2015)
- [Right-wing German Hate Speech on Twitter: Analysis and Automatic Detection](https://mlsanity.com/paper/1910.07518/) (2019)
- ["Like Sheep Among Wolves": Characterizing Hateful Users on Twitter](https://mlsanity.com/paper/1801.00317/) (2017)
- [UM-IU@LING at SemEval-2019 Task 6: Identifying Offensive Tweets Using BERT and SVMs](https://mlsanity.com/paper/1904.03450/) (2019)
- [Automatic Detection of Online Jihadist Hate Speech](https://mlsanity.com/paper/1803.04596/) (2018)

## Built on by (most cited first, 26 of 26)

- [Explainability in Practice: A Survey of Explainable NLP Across Various Domains](https://mlsanity.com/paper/2502.00837/) (2025)
- [Distilling Knowledge from Large Language Models: A Concept Bottleneck Model for Hate and Counter Speech Recognition](https://mlsanity.com/paper/2508.08274/) (2025)
- [Adaptable Moral Stances of Large Language Models on Sexist Content: Implications for Society and Gender Discourse](https://mlsanity.com/paper/2410.00175/) (2024)
- [Social Support Detection from Social Media Texts](https://mlsanity.com/paper/2411.02580/) (2024)
- [A survey of textual cyber abuse detection using cutting-edge language models and large language models](https://mlsanity.com/paper/2501.05443/) (2025)
- [Efficient Hate Speech Detection: Evaluating 38 Models from Traditional Methods to Transformers](https://mlsanity.com/paper/2509.14266/) (2025)
- [A Survey on Automatic Online Hate Speech Detection in Low-Resource Languages](https://mlsanity.com/paper/2411.19017/) (2024)
- [Culture Matters in Toxic Language Detection in Persian](https://mlsanity.com/paper/2506.03458/) (2025)
- [HP-BERT: A framework for longitudinal study of Hinduphobia on social media via language models](https://mlsanity.com/paper/2501.05482/) (2025)
- [EdgeAIGuard: Agentic LLMs for Minor Protection in Digital Spaces](https://mlsanity.com/paper/2503.00092/) (2025)
- [The Enemy from Within: A Study of Political Delegitimization Discourse in Israeli Political Speech](https://mlsanity.com/paper/2508.15524/) (2025)
- [Specializing General-purpose LLM Embeddings for Implicit Hate Speech Detection across Datasets](https://mlsanity.com/paper/2508.20750/) (2025)
- [Causality Guided Representation Learning for Cross-Style Hate Speech Detection](https://mlsanity.com/paper/2510.07707/) (2025)
- [Speech Detection Task Against Asian Hate: BERT the Central, While Data-Centric Studies the Crucial](https://mlsanity.com/paper/2206.02114/) (2022)
- [Transferring Knowledge via Neighborhood-Aware Optimal Transport for Low-Resource Hate Speech Detection](https://mlsanity.com/paper/2210.09340/) (2022)
- [EvoText: Enhancing Natural Language Generation Models via Self-Escalation Learning for Up-to-Date Knowledge and Improved Performance](https://mlsanity.com/paper/2302.03896/) (2023)
- [Exploratory Data Analysis on Code-mixed Misogynistic Comments](https://mlsanity.com/paper/2403.09709/) (2024)
- [Unveiling Social Media Comments with a Novel Named Entity Recognition System for Identity Groups](https://mlsanity.com/paper/2405.13011/) (2024)
- [Empirical Evaluation of Public HateSpeech Datasets](https://mlsanity.com/paper/2407.12018/) (2024)
- [Code-Mixed Telugu-English Hate Speech Detection](https://mlsanity.com/paper/2502.10632/) (2025)
- [Detoxify: A framework for abusive text transformation using LLMs](https://mlsanity.com/paper/2507.10177/) (2025)
- [From BERT to Qwen: Hate Detection across architectures](https://mlsanity.com/paper/2507.10468/) (2025)
- [Ethic-BERT: An Enhanced Deep Learning Model for Ethical and Non-Ethical Content Classification](https://mlsanity.com/paper/2510.12850/) (2025)
- [Feature Selection Empowered BERT for Detection of Hate Speech with Vocabulary Augmentation](https://mlsanity.com/paper/2512.02141/) (2025)
- [FAD-SA-GRU: Enhancing Hate Speech Detection in Algerian Dialect Through Feature-Augmented Self-Attention GRU Networks](https://mlsanity.com/paper/2607.11279/) (2026)
- [Efficient Adaptation of LLMs for Hate Speech Detection in Low-Resource Languages: A Comparative Study on Roman Urdu](https://mlsanity.com/paper/2608.18142/) (2026)

## Abstract (arXiv)

The enormous amount of data being generated on the web and social media has increased the demand for detecting online hate speech. Detecting hate speech will reduce their negative impact and influence on others. A lot of effort in the Natural Language Processing (NLP) domain aimed to detect hate speech in general or detect specific hate speech such as religion, race, gender, or sexual orientation. Hate communities tend to use abbreviations, intentional spelling mistakes, and coded words in their communication to evade detection, adding more challenges to hate speech detection tasks. Thus, word representation will play an increasingly pivotal role in detecting hate speech. This paper investigates the feasibility of leveraging domain-specific word embedding in Bidirectional LSTM based deep model to automatically detect/classify hate speech. Furthermore, we investigate the use of the transfer learning language model (BERT) on hate speech problem as a binary classification task. The experiments showed that domainspecific word embedding with the Bidirectional LSTM based deep model achieved a 93% f1-score while BERT achieved up to 96% f1-score on a combined balanced dataset from available hate speech datasets.