# BERT Transformer model for Detecting Arabic GPT2 Auto-Generated Tweets

arXiv: https://arxiv.org/abs/2101.09345 · HTML: https://mlsanity.com/paper/2101.09345/

## Key facts

- Cited by 4 papers in the map (data through 14 Oct 2026).
- Builds on 9 of its 19 references that are in the map.
- First posted on arXiv 22 Jan 2021.

## Builds on (most cited first, 9 of 9)

- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [AraBERT: Transformer-based Model for Arabic Language Understanding](https://mlsanity.com/paper/2003.00104/) (2020)
- [Online Human-Bot Interactions: Detection, Estimation, and Characterization](https://mlsanity.com/paper/1703.03107/) (2017)
- [Natural Language Processing Advancements By Deep Learning: A Survey](https://mlsanity.com/paper/2003.01200/) (2020)
- [Deep Neural Networks for Bot Detection](https://mlsanity.com/paper/1802.04289/) (2018)
- [Twitter Bot Detection Using Bidirectional Long Short-term Memory Neural Networks and Word Embeddings](https://mlsanity.com/paper/2002.01336/) (2020)
- [Seminar Users in the Arabic Twitter Sphere](https://mlsanity.com/paper/1707.07276/) (2017)
- [Instagram Fake and Automated Account Detection](https://mlsanity.com/paper/1910.03090/) (2019)
- [Using Deep Networks and Transfer Learning to Address Disinformation](https://mlsanity.com/paper/1905.10412/) (2019)

## Built on by (most cited first, 4 of 4)

- [Seq vs Seq: An Open Suite of Paired Encoders and Decoders](https://mlsanity.com/paper/2507.11412/) (2025)
- [Data Augmentation using Transformers and Similarity Measures for Improving Arabic Text Classification](https://mlsanity.com/paper/2212.13939/) (2022)
- [ALHD: A Large-Scale and Multigenre Benchmark Dataset for Arabic LLM-Generated Text Detection](https://mlsanity.com/paper/2510.03502/) (2025)
- [CASPER in the Machine: Insights into Character Variety in LLM-Generated Stories](https://mlsanity.com/paper/2606.22454/) (2026)

## Abstract (arXiv)

During the last two decades, we have progressively turned to the Internet and social media to find news, entertain conversations and share opinion. Recently, OpenAI has developed a ma-chine learning system called GPT-2 for Generative Pre-trained Transformer-2, which can pro-duce deepfake texts. It can generate blocks of text based on brief writing prompts that look like they were written by humans, facilitating the spread false or auto-generated text. In line with this progress, and in order to counteract potential dangers, several methods have been pro-posed for detecting text written by these language models. In this paper, we propose a transfer learning based model that will be able to detect if an Arabic sentence is written by humans or automatically generated by bots. Our dataset is based on tweets from a previous work, which we have crawled and extended using the Twitter API. We used GPT2-Small-Arabic to generate fake Arabic Sentences. For evaluation, we compared different recurrent neural network (RNN) word embeddings based baseline models, namely: LSTM, BI-LSTM, GRU and BI-GRU, with a transformer-based model. Our new transfer-learning model has obtained an accuracy up to 98%. To the best of our knowledge, this work is the first study where ARABERT and GPT2 were combined to detect and classify the Arabic auto-generated texts.