# The Arabic AI Fingerprint: Stylometric Analysis and Detection of Large Language Models Text

arXiv: https://arxiv.org/abs/2505.23276 · HTML: https://mlsanity.com/paper/2505.23276/

## Key facts

- Cited by 3 papers in the map (data through 14 Oct 2026).
- Builds on 23 of its 39 references that are in the map.
- First posted on arXiv 29 May 2025.

## Builds on (most cited first, 23 of 23)

- [Language Models are Few-Shot Learners](https://mlsanity.com/paper/2005.14165/) (2020)
- [GPT-4 Technical Report](https://mlsanity.com/paper/2303.08774/) (2023)
- [The Llama 3 Herd of Models](https://mlsanity.com/paper/2407.21783/) (2024)
- [LLaMA: Open and Efficient Foundation Language Models](https://mlsanity.com/paper/2302.13971/) (2023)
- [Unsupervised Cross-lingual Representation Learning at Scale](https://mlsanity.com/paper/1911.02116/) (2019)
- [DetectGPT: Zero-Shot Machine-Generated Text Detection using Probability Curvature](https://mlsanity.com/paper/2301.11305/) (2023)
- [GPT-NeoX-20B: An Open-Source Autoregressive Language Model](https://mlsanity.com/paper/2204.06745/) (2022)
- [Textbooks Are All You Need](https://mlsanity.com/paper/2306.11644/) (2023)
- [How Close is ChatGPT to Human Experts? Comparison Corpus, Evaluation, and Detection](https://mlsanity.com/paper/2301.07597/) (2023)
- [Fast-DetectGPT: Efficient Zero-Shot Detection of Machine-Generated Text via Conditional Probability Curvature](https://mlsanity.com/paper/2310.05130/) (2023)
- [MAGE: Machine-generated Text Detection in the Wild](https://mlsanity.com/paper/2305.13242/) (2023)
- [Mapping the Increasing Use of LLMs in Scientific Papers](https://mlsanity.com/paper/2404.01268/) (2024)
- [OUTFOX: LLM-Generated Essay Detection Through In-Context Learning with Adversarially Generated Examples](https://mlsanity.com/paper/2307.11729/) (2023)
- [AceGPT, Localizing Large Language Models in Arabic](https://mlsanity.com/paper/2309.12053/) (2023)
- [A Survey on Detection of LLMs-Generated Content](https://mlsanity.com/paper/2310.15654/) (2023)
- [Jais and Jais-chat: Arabic-Centric Foundation and Instruction-Tuned Open Generative Large Language Models](https://mlsanity.com/paper/2308.16149/) (2023)
- [Contrasting Linguistic Patterns in Human and LLM-Generated News Text](https://mlsanity.com/paper/2308.09067/) (2023)
- [Stylometric Detection of AI-Generated Text in Twitter Timelines](https://mlsanity.com/paper/2303.03697/) (2023)
- [ALLaM: Large Language Models for Arabic and English](https://mlsanity.com/paper/2407.15390/) (2024)
- [AraELECTRA: Pre-Training Text Discriminators for Arabic Language Understanding](https://mlsanity.com/paper/2012.15516/) (2020)
- [CHEAT: A Large-scale Dataset for Detecting ChatGPT-writtEn AbsTracts](https://mlsanity.com/paper/2304.12008/) (2023)
- [HC3 Plus: A Semantic-Invariant Human ChatGPT Comparison Corpus](https://mlsanity.com/paper/2309.02731/) (2023)
- [Evaluating Computational Language Models with Scaling Properties of Natural Language](https://mlsanity.com/paper/1906.09379/) (2019)

## Built on by (most cited first, 3 of 3)

- [AI Text Detectors and the Misclassification of Slightly Polished Arabic Text](https://mlsanity.com/paper/2511.16690/) (2025)
- [Stylometry recognizes human and LLM-generated texts in short samples](https://mlsanity.com/paper/2507.00838/) (2025)
- [ALHD: A Large-Scale and Multigenre Benchmark Dataset for Arabic LLM-Generated Text Detection](https://mlsanity.com/paper/2510.03502/) (2025)

## Abstract (arXiv)

Large Language Models (LLMs) have achieved unprecedented capabilities in generating human-like text, posing subtle yet significant challenges for information integrity across critical domains, including education, social media, and academia, enabling sophisticated misinformation campaigns, compromising healthcare guidance, and facilitating targeted propaganda. This challenge becomes severe, particularly in under-explored and low-resource languages like Arabic. This paper presents a comprehensive investigation of Arabic machine-generated text, examining multiple generation strategies (generation from the title only, content-aware generation, and text refinement) across diverse model architectures (ALLaM, Jais, Llama, and GPT-4) in academic, and social media domains. Our stylometric analysis reveals distinctive linguistic patterns differentiating human-written from machine-generated Arabic text across these varied contexts. Despite their human-like qualities, we demonstrate that LLMs produce detectable signatures in their Arabic outputs, with domain-specific characteristics that vary significantly between different contexts. Based on these insights, we developed BERT-based detection models that achieved exceptional performance in formal contexts (up to 99.9% F1-score) with strong precision across model architectures. Our cross-domain analysis confirms generalization challenges previously reported in the literature. To the best of our knowledge, this work represents the most comprehensive investigation of Arabic machine-generated text to date, uniquely combining multiple prompt generation methods, diverse model architectures, and in-depth stylometric analysis across varied textual domains, establishing a foundation for developing robust, linguistically-informed detection systems essential for preserving information integrity in Arabic-language contexts.