# Transformers and Ensemble methods: A solution for Hate Speech Detection in Arabic languages

arXiv: https://arxiv.org/abs/2303.09823 · HTML: https://mlsanity.com/paper/2303.09823/

## Key facts

- Cited by 2 papers in the map (data through 14 Oct 2026).
- Builds on 12 of its 26 references that are in the map.
- First posted on arXiv 17 Mar 2023.

## Builds on (most cited first, 12 of 12)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [Unsupervised Cross-lingual Representation Learning at Scale](https://mlsanity.com/paper/1911.02116/) (2019)
- [A Survey of Transformers](https://mlsanity.com/paper/2106.04554/) (2021)
- [AraBERT: Transformer-based Model for Arabic Language Understanding](https://mlsanity.com/paper/2003.00104/) (2020)
- [SemEval-2020 Task 12: Multilingual Offensive Language Identification in Social Media (OffensEval 2020)](https://mlsanity.com/paper/2006.07235/) (2020)
- [AraGPT2: Pre-Trained Transformer for Arabic Language Generation](https://mlsanity.com/paper/2012.15520/) (2020)
- [AraELECTRA: Pre-Training Text Discriminators for Arabic Language Understanding](https://mlsanity.com/paper/2012.15516/) (2020)
- [Emojis as Anchors to Detect Arabic Offensive Language and Hate Speech](https://mlsanity.com/paper/2201.06723/) (2022)
- [AI-UPV at IberLEF-2021 DETOXIS task: Toxicity Detection in Immigration-Related Web News Comments Using Transformers and Statistical Models](https://mlsanity.com/paper/2111.04530/) (2021)
- [Sexism Prediction in Spanish and English Tweets Using Monolingual and Multilingual BERT and Ensemble Models](https://mlsanity.com/paper/2111.04551/) (2021)
- [Deep Multi-Task Models for Misogyny Identification and Categorization on Arabic Social Media](https://mlsanity.com/paper/2206.08407/) (2022)

## Built on by (most cited first, 2 of 2)

- [Toxic language detection: a systematic review of Arabic datasets](https://mlsanity.com/paper/2312.07228/) (2023)
- [AI-UPV at EXIST 2023 -- Sexism Characterization Using Large Language Models Under The Learning with Disagreements Regime](https://mlsanity.com/paper/2307.03385/) (2023)

## Abstract (arXiv)

This paper describes our participation in the shared task of hate speech detection, which is one of the subtasks of the CERIST NLP Challenge 2022. Our experiments evaluate the performance of six transformer models and their combination using 2 ensemble approaches. The best results on the training set, in a five-fold cross validation scenario, were obtained by using the ensemble approach based on the majority vote. The evaluation of this approach on the test set resulted in an F1-score of 0.60 and an Accuracy of 0.86.