# Hate speech detection using static BERT embeddings

arXiv: https://arxiv.org/abs/2106.15537 · HTML: https://mlsanity.com/paper/2106.15537/

## Key facts

- Cited by 4 papers in the map (data through 14 Oct 2026).
- Builds on 9 of its 30 references that are in the map.
- First posted on arXiv 29 Jun 2021.

## Builds on (most cited first, 9 of 9)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [Neural Machine Translation by Jointly Learning to Align and Translate](https://mlsanity.com/paper/1409.0473/) (2014)
- [Learning Phrase Representations using RNN Encoder-Decoder for Statistical Machine Translation](https://mlsanity.com/paper/1406.1078/) (2014)
- [Distributed Representations of Words and Phrases and their Compositionality](https://mlsanity.com/paper/1310.4546/) (2013)
- [Automated Hate Speech Detection and the Problem of Offensive Language](https://mlsanity.com/paper/1703.04009/) (2017)
- [Deep Learning for Hate Speech Detection in Tweets](https://mlsanity.com/paper/1706.00188/) (2017)
- [Automated diagnosis of COVID-19 with limited posteroanterior chest X-ray images using fine-tuned deep neural networks](https://mlsanity.com/paper/2004.11676/) (2020)
- [BERT-Based Sentiment Analysis: A Software Engineering Perspective](https://mlsanity.com/paper/2106.02581/) (2021)

## Built on by (most cited first, 4 of 4)

- [A survey of textual cyber abuse detection using cutting-edge language models and large language models](https://mlsanity.com/paper/2501.05443/) (2025)
- [Speech Detection Task Against Asian Hate: BERT the Central, While Data-Centric Studies the Crucial](https://mlsanity.com/paper/2206.02114/) (2022)
- [Non-Contextual BERT or FastText? A Comparative Analysis](https://mlsanity.com/paper/2411.17661/) (2024)
- [Efficient Adaptation of LLMs for Hate Speech Detection in Low-Resource Languages: A Comparative Study on Roman Urdu](https://mlsanity.com/paper/2608.18142/) (2026)

## Abstract (arXiv)

With increasing popularity of social media platforms hate speech is emerging as a major concern, where it expresses abusive speech that targets specific group characteristics, such as gender, religion or ethnicity to spread violence. Earlier people use to verbally deliver hate speeches but now with the expansion of technology, some people are deliberately using social media platforms to spread hate by posting, sharing, commenting, etc. Whether it is Christchurch mosque shootings or hate crimes against Asians in west, it has been observed that the convicts are very much influenced from hate text present online. Even though AI systems are in place to flag such text but one of the key challenges is to reduce the false positive rate (marking non hate as hate), so that these systems can detect hate speech without undermining the freedom of expression. In this paper, we use ETHOS hate speech detection dataset and analyze the performance of hate speech detection classifier by replacing or integrating the word embeddings (fastText (FT), GloVe (GV) or FT + GV) with static BERT embeddings (BE). With the extensive experimental trails it is observed that the neural network performed better with static BE compared to using FT, GV or FT + GV as word embeddings. In comparison to fine-tuned BERT, one metric that significantly improved is specificity.