# Offensive Hebrew Corpus and Detection using BERT

arXiv: https://arxiv.org/abs/2309.02724 · HTML: https://mlsanity.com/paper/2309.02724/

## Key facts

- Cited by 6 papers in the map (data through 14 Oct 2026).
- Builds on 19 of its 38 references that are in the map.
- First posted on arXiv 6 Sep 2023.

## Builds on (most cited first, 19 of 19)

- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [Unsupervised Cross-lingual Representation Learning at Scale](https://mlsanity.com/paper/1911.02116/) (2019)
- [AraBERT: Transformer-based Model for Arabic Language Understanding](https://mlsanity.com/paper/2003.00104/) (2020)
- [Predicting the Type and Target of Offensive Posts in Social Media](https://mlsanity.com/paper/1902.09666/) (2019)
- [SemEval-2019 Task 6: Identifying and Categorizing Offensive Language in Social Media (OffensEval)](https://mlsanity.com/paper/1903.08983/) (2019)
- [SemEval-2020 Task 12: Multilingual Offensive Language Identification in Social Media (OffensEval 2020)](https://mlsanity.com/paper/2006.07235/) (2020)
- [Understanding Abuse: A Typology of Abusive Language Detection Subtasks](https://mlsanity.com/paper/1705.09899/) (2017)
- [A Panoramic Survey of Natural Language Processing in the Arab World](https://mlsanity.com/paper/2011.12631/) (2020)
- [Pre-Training BERT on Arabic Tweets: Practical Considerations](https://mlsanity.com/paper/2102.10684/) (2021)
- [Wojood: Nested Arabic Named Entity Corpus and Recognition using BERT](https://mlsanity.com/paper/2205.09651/) (2022)
- [Arabic Offensive Language on Twitter: Analysis and Experiments](https://mlsanity.com/paper/2004.02192/) (2020)
- [AlephBERT:A Hebrew Large Pre-Trained Language Model to Start-off your Hebrew NLP Application With](https://mlsanity.com/paper/2104.04052/) (2021)
- [Curras + Baladi: Towards a Levantine Corpus](https://mlsanity.com/paper/2205.09692/) (2022)
- [The Arabic Ontology -- An Arabic Wordnet with Ontologically Clean Content](https://mlsanity.com/paper/2205.09664/) (2022)
- [ArabGlossBERT: Fine-Tuning BERT on Context-Gloss Pairs for WSD](https://mlsanity.com/paper/2205.09685/) (2022)
- [HeBERT & HebEMO: a Hebrew BERT Model and a Tool for Polarity Analysis and Emotion Recognition](https://mlsanity.com/paper/2102.01909/) (2021)
- [Emojis as Anchors to Detect Arabic Offensive Language and Hate Speech](https://mlsanity.com/paper/2201.06723/) (2022)
- [NLPDove at SemEval-2020 Task 12: Improving Offensive Language Detection with Cross-lingual Transfer](https://mlsanity.com/paper/2008.01354/) (2020)
- [WOLI at SemEval-2020 Task 12: Arabic Offensive Language Identification on Different Twitter Datasets](https://mlsanity.com/paper/2009.05456/) (2020)

## Built on by (most cited first, 6 of 6)

- [RTP-LX: Can LLMs Evaluate Toxicity in Multilingual Scenarios?](https://mlsanity.com/paper/2404.14397/) (2024)
- [The FIGNEWS Shared Task on News Media Narratives](https://mlsanity.com/paper/2407.18147/) (2024)
- [Sina at FigNews 2024: Multilingual Datasets Annotated with Bias and Propaganda](https://mlsanity.com/paper/2407.09327/) (2024)
- [Translate, then Detect: Leveraging Machine Translation for Cross-Lingual Toxicity Classification](https://mlsanity.com/paper/2509.14493/) (2025)
- [SinaTools: Open Source Toolkit for Arabic Natural Language Processing](https://mlsanity.com/paper/2411.01523/) (2024)
- [When Explanations Cannot Be Read: Measuring and Correcting SHAP and LIME Rendering for Right-to-Left Languages](https://mlsanity.com/paper/2609.28565/) (2026)

## Abstract (arXiv)

Offensive language detection has been well studied in many languages, but it is lagging behind in low-resource languages, such as Hebrew. In this paper, we present a new offensive language corpus in Hebrew. A total of 15,881 tweets were retrieved from Twitter. Each was labeled with one or more of five classes (abusive, hate, violence, pornographic, or none offensive) by Arabic-Hebrew bilingual speakers. The annotation process was challenging as each annotator is expected to be familiar with the Israeli culture, politics, and practices to understand the context of each tweet. We fine-tuned two Hebrew BERT models, HeBERT and AlephBERT, using our proposed dataset and another published dataset. We observed that our data boosts HeBERT performance by 2% when combined with D_OLaH. Fine-tuning AlephBERT on our data and testing on D_OLaH yields 69% accuracy, while fine-tuning on D_OLaH and testing on our data yields 57% accuracy, which may be an indication to the generalizability our data offers. Our dataset and fine-tuned models are available on GitHub and Huggingface.