# A BERT-Based Transfer Learning Approach for Hate Speech Detection in Online Social Media

arXiv: https://arxiv.org/abs/1910.12574 · HTML: https://mlsanity.com/paper/1910.12574/

## Key facts

- Cited by 96 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 28 Oct 2019.

## Built on by (most cited first, 30 of 96)

- [Dealing with Disagreements: Looking Beyond the Majority Vote in Subjective Annotations](https://mlsanity.com/paper/2110.05719/) (2021)
- [Towards generalisable hate speech detection: a review on obstacles and solutions](https://mlsanity.com/paper/2102.08886/) (2021)
- [How Hate Speech Varies by Target Identity: A Computational Analysis](https://mlsanity.com/paper/2210.10839/) (2022)
- [Deep Learning for Hate Speech Detection: A Comparative Study](https://mlsanity.com/paper/2202.09517/) (2022)
- [Memotion 3: Dataset on Sentiment and Emotion Analysis of Codemixed Hindi-English Memes](https://mlsanity.com/paper/2303.09892/) (2023)
- [Explainable Abuse Detection as Intent Classification and Slot Filling](https://mlsanity.com/paper/2210.02659/) (2022)
- [Generalizing Hate Speech Detection Using Multi-Task Learning: A Case Study of Political Public Figures](https://mlsanity.com/paper/2208.10598/) (2022)
- [Vietnamese Hate and Offensive Detection using PhoBERT-CNN and Social Media Streaming Data](https://mlsanity.com/paper/2206.00524/) (2022)
- [Deciphering Implicit Hate: Evaluating Automated Detection Algorithms for Multimodal Hate](https://mlsanity.com/paper/2106.05903/) (2021)
- [A Computational Approach to Understand Mental Health from Reddit: Knowledge-aware Multitask Learning Framework](https://mlsanity.com/paper/2203.11856/) (2022)
- [Transfer Language Selection for Zero-Shot Cross-Lingual Abusive Language Detection](https://mlsanity.com/paper/2206.00962/) (2022)
- [Target Span Detection for Implicit Harmful Content](https://mlsanity.com/paper/2403.19836/) (2024)
- [One to rule them all: Towards Joint Indic Language Hate Speech Detection](https://mlsanity.com/paper/2109.13711/) (2021)
- [Predicting Hate Intensity of Twitter Conversation Threads](https://mlsanity.com/paper/2206.08406/) (2022)
- [Revisiting Hate Speech Benchmarks: From Data Curation to System Deployment](https://mlsanity.com/paper/2306.01105/) (2023)
- ["Define Your Terms" : Enhancing Efficient Offensive Speech Classification with Definition](https://mlsanity.com/paper/2402.03221/) (2024)
- [HateTinyLLM : Hate Speech Detection Using Tiny Large Language Models](https://mlsanity.com/paper/2405.01577/) (2024)
- [Hidden behind the obvious: misleading keywords and implicitly abusive language on social media](https://mlsanity.com/paper/2205.01374/) (2022)
- [Understanding Social Media Cross-Modality Discourse in Linguistic Space](https://mlsanity.com/paper/2302.13311/) (2023)
- [Multi-Modal Discussion Transformer: Integrating Text, Images and Graph Transformers to Detect Hate Speech on Social Media](https://mlsanity.com/paper/2307.09312/) (2023)
- [User-Aware Multilingual Abusive Content Detection in Social Media](https://mlsanity.com/paper/2410.21321/) (2024)
- [Exploring multi-task multi-lingual learning of transformer models for hate speech and offensive speech identification in social media](https://mlsanity.com/paper/2101.11155/) (2021)
- [Sexism Prediction in Spanish and English Tweets Using Monolingual and Multilingual BERT and Ensemble Models](https://mlsanity.com/paper/2111.04551/) (2021)
- [T5 for Hate Speech, Augmented Data and Ensemble](https://mlsanity.com/paper/2210.05480/) (2022)
- [Hate Speech and Offensive Language Detection using an Emotion-aware Shared Encoder](https://mlsanity.com/paper/2302.08777/) (2023)
- [LionGuard: Building a Contextualized Moderation Classifier to Tackle Localized Unsafe Content](https://mlsanity.com/paper/2407.10995/) (2024)
- [Adversarial Attacks and Defenses for Social Network Text Processing Applications: Techniques, Challenges and Future Research Directions](https://mlsanity.com/paper/2110.13980/) (2021)
- ["Stop Asian Hate!" : Refining Detection of Anti-Asian Hate Speech During the COVID-19 Pandemic](https://mlsanity.com/paper/2112.02265/) (2021)
- [Is More Data Better? Re-thinking the Importance of Efficiency in Abusive Language Detection with Transformers-Based Active Learning](https://mlsanity.com/paper/2209.10193/) (2022)
- [Hate Speech and Offensive Content Detection in Indo-Aryan Languages: A Battle of LSTM and Transformers](https://mlsanity.com/paper/2312.05671/) (2023)

## Abstract (arXiv)

Generated hateful and toxic content by a portion of users in social media is a rising phenomenon that motivated researchers to dedicate substantial efforts to the challenging direction of hateful content identification. We not only need an efficient automatic hate speech detection model based on advanced machine learning and natural language processing, but also a sufficiently large amount of annotated data to train a model. The lack of a sufficient amount of labelled hate speech data, along with the existing biases, has been the main issue in this domain of research. To address these needs, in this study we introduce a novel transfer learning approach based on an existing pre-trained language model called BERT (Bidirectional Encoder Representations from Transformers). More specifically, we investigate the ability of BERT at capturing hateful context within social media content by using new fine-tuning methods based on transfer learning. To evaluate our proposed approach, we use two publicly available datasets that have been annotated for racism, sexism, hate, or offensive content on Twitter. The results show that our solution obtains considerable performance on these datasets in terms of precision and recall in comparison to existing approaches. Consequently, our model can capture some biases in data annotation and collection process and can potentially lead us to a more accurate model.