# Transfer Learning Approach for Arabic Offensive Language Detection System -- BERT-Based Model

arXiv: https://arxiv.org/abs/2102.05708 · HTML: https://mlsanity.com/paper/2102.05708/

## Key facts

- Cited by 2 papers in the map (data through 7 Oct 2026).
- Builds on 6 of its 21 references that are in the map.
- First posted on arXiv 9 Feb 2021.

## Builds on (most cited first, 6 of 6)

- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [AraBERT: Transformer-based Model for Arabic Language Understanding](https://mlsanity.com/paper/2003.00104/) (2020)
- [KUISAIL at SemEval-2020 Task 12: BERT-CNN for Offensive Speech Identification in Social Media](https://mlsanity.com/paper/2007.13184/) (2020)
- [A Unified Deep Learning Architecture for Abuse Detection](https://mlsanity.com/paper/1802.00385/) (2018)
- [OSACT4 Shared Task on Offensive Language Detection: Intensive Preprocessing-Based Approach](https://mlsanity.com/paper/2005.07297/) (2020)
- [Leveraging Affective Bidirectional Transformers for Offensive Language Detection](https://mlsanity.com/paper/2006.01266/) (2020)

## Built on by (most cited first, 2 of 2)

- [Ensemble of pre-trained language models and data augmentation for hate speech detection from Arabic tweets](https://mlsanity.com/paper/2407.02448/) (2024)
- [Arabic Hate Speech Identification and Masking in Social Media using Deep Learning Models and Pre-trained Models Fine-tuning](https://mlsanity.com/paper/2507.23661/) (2025)

## Abstract (arXiv)

Developing a system to detect online offensive language is very important to the health and the security of online users. Studies have shown that cyberhate, online harassment and other misuses of technology are on the rise, particularly during the global Coronavirus pandemic in 2020. According to the latest report by the Anti-Defamation League (ADL), 35% of online users reported online harassment related to their identity-based characteristics, which is a 3% increase over 2019. Applying advanced techniques from the Natural Language Processing (NLP) field to support the development of an online hate-free community is a critical task for social justice. Transfer learning enhances the performance of the classifier by allowing the transfer of knowledge from one domain or one dataset to others that have not been seen before, thus, supporting the classifier to be more generalizable. In our study, we apply the principles of transfer learning cross multiple Arabic offensive language datasets to compare the effects on system performance. This study aims at investigating the effects of fine-tuning and training Bidirectional Encoder Representations from Transformers (BERT) model on multiple Arabic offensive language datasets individually and testing it using other datasets individually. Our experiment starts with a comparison among multiple BERT models to guide the selection of the main model that is used for our study. The study also investigates the effects of concatenating all datasets to be used for fine-tuning and training BERT model. Our results demonstrate the limited effects of transfer learning on the performance of the classifiers, particularly for highly dialectic comments.