# 1.5 billion words Arabic Corpus

arXiv: https://arxiv.org/abs/1611.04033 · HTML: https://mlsanity.com/paper/1611.04033/

## Key facts

- Cited by 35 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 12 Nov 2016.

## Built on by (most cited first, 30 of 35)

- [Having Beer after Prayer? Measuring Cultural Bias in Large Language Models](https://mlsanity.com/paper/2305.14456/) (2023)
- [The BigScience ROOTS Corpus: A 1.6TB Composite Multilingual Dataset](https://mlsanity.com/paper/2303.03915/) (2023)
- [The Interplay of Variant, Size, and Task Type in Arabic Pre-trained Language Models](https://mlsanity.com/paper/2103.06678/) (2021)
- [AraT5: Text-to-Text Transformers for Arabic Language Generation](https://mlsanity.com/paper/2109.12068/) (2021)
- [ORCA: A Challenging Benchmark for Arabic Language Understanding](https://mlsanity.com/paper/2212.10758/) (2022)
- [Pre-Training BERT on Arabic Tweets: Practical Considerations](https://mlsanity.com/paper/2102.10684/) (2021)
- [Wojood: Nested Arabic Named Entity Corpus and Recognition using BERT](https://mlsanity.com/paper/2205.09651/) (2022)
- [Peacock: A Family of Arabic Multimodal Large Language Models and Benchmarks](https://mlsanity.com/paper/2403.01031/) (2024)
- [ZEN 2.0: Continue Training and Adaption for N-gram Enhanced Text Encoders](https://mlsanity.com/paper/2105.01279/) (2021)
- [Everything Is All It Takes: A Multipronged Strategy for Zero-Shot Cross-Lingual Information Extraction](https://mlsanity.com/paper/2109.06798/) (2021)
- [Evaluating Various Tokenizers for Arabic Text Classification](https://mlsanity.com/paper/2106.07540/) (2021)
- [JASMINE: Arabic GPT Models for Few-Shot Learning](https://mlsanity.com/paper/2212.10755/) (2022)
- [Swan and ArabicMTEB: Dialect-Aware, Arabic-Centric, Cross-Lingual, and Cross-Cultural Embedding Models and Benchmarks](https://mlsanity.com/paper/2411.01192/) (2024)
- [A Transformer-based Approach for Arabic Offline Handwritten Text Recognition](https://mlsanity.com/paper/2307.15045/) (2023)
- [A Parameter-Efficient Learning Approach to Arabic Dialect Identification with Pre-Trained General-Purpose Speech Model](https://mlsanity.com/paper/2305.11244/) (2023)
- [ArBanking77: Intent Detection Neural Model and a New Dataset in Modern and Dialectical Arabic](https://mlsanity.com/paper/2310.19034/) (2023)
- [AraPoemBERT: A Pretrained Language Model for Arabic Poetry Analysis](https://mlsanity.com/paper/2403.12392/) (2024)
- [Revisiting Entropy Rate Constancy in Text](https://mlsanity.com/paper/2305.12084/) (2023)
- [AraMUS: Pushing the Limits of Data and Model Scale for Arabic Natural Language Processing](https://mlsanity.com/paper/2306.06800/) (2023)
- [SaudiBERT: A Large Language Model Pretrained on Saudi Dialect Corpora](https://mlsanity.com/paper/2405.06239/) (2024)
- [On the Role of Images for Analyzing Claims in Social Media](https://mlsanity.com/paper/2103.09602/) (2021)
- [Octopus: A Multitask Model and Toolkit for Arabic Natural Language Generation](https://mlsanity.com/paper/2310.16127/) (2023)
- [EgyBERT: A Large Language Model Pretrained on Egyptian Dialect Corpora](https://mlsanity.com/paper/2408.03524/) (2024)
- [Transformers to Fight the COVID-19 Infodemic](https://mlsanity.com/paper/2104.12201/) (2021)
- [Revisiting Pre-trained Language Models and their Evaluation for Arabic Natural Language Understanding](https://mlsanity.com/paper/2205.10687/) (2022)
- [Revisiting Data Compression with Language Modeling](https://mlsanity.com/paper/2601.02875/) (2026)
- [Post-hoc analysis of Arabic transformer models](https://mlsanity.com/paper/2210.09990/) (2022)
- [Ashaar: Automatic Analysis and Generation of Arabic Poetry Using Deep Learning Approaches](https://mlsanity.com/paper/2307.06218/) (2023)
- [Towards the Development of Balanced Synthetic Data for Correcting Grammatical Errors in Arabic: An Approach Based on Error Tagging Model and Synthetic Data Generating Model](https://mlsanity.com/paper/2502.05312/) (2025)
- [New Arabic Medical Dataset for Diseases Classification](https://mlsanity.com/paper/2106.15236/) (2021)

## Abstract (arXiv)

This study is an attempt to build a contemporary linguistic corpus for Arabic language. The corpus produced, is a text corpus includes more than five million newspaper articles. It contains over a billion and a half words in total, out of which, there is about three million unique words. The data were collected from newspaper articles in ten major news sources from eight Arabic countries, over a period of fourteen years. The corpus was encoded with two types of encoding, namely: UTF-8, and Windows CP-1256. Also it was marked with two mark-up languages, namely: SGML, and XML.