naab: A ready-to-use plug-and-play corpus for Farsi
Not on the map. Why?
Abstract (arXiv)
The rise of large language models (LLMs) has transformed numerous natural language processing (NLP) tasks, yet their performance in low and mid-resource languages, such as Farsi, still lags behind resource-rich languages like English. To address this gap, we introduce naab, the largest publicly available, cleaned, and ready-to-use Farsi textual corpus. naab consists of 130GB of data, comprising over 250 million paragraphs and 15 billion words. Named after the Farsi word NAAB (meaning "pure" or "high-grade"), this corpus is openly accessible via Hugging Face, offering researchers a valuable resource for Farsi NLP tasks. In addition to naab, we provide naab-raw, an unprocessed version of the dataset, along with a pre-processing toolkit that allows users to clean their custom corpora. These resources empower NLP researchers and practitioners, particularly those focusing on low-resource languages, to improve the performance of LLMs in their respective domains and bridge the gap between resource-rich and resource-poor languages.
Key facts. Cited by 5 papers in the map (data through 14 Oct 2026). Builds on 10 of its 32 references that are in the map. First posted on arXiv 29 Aug 2022.
Builds on
10 of 32 references.
References · 10
- 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,399
- 2019Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformercited by 14,599
- 2019BART: Denoising Sequence-to-Sequence Pre-training for Natural Language Generation, Translation, and Comprehensioncited by 5,893
- 2022Beyond the Imitation Game: Quantifying and extrapolating the capabilities of language modelscited by 1,998
- 2021Datasets: A Community Library for Natural Language Processingcited by 530
- 2022Are Large Pre-Trained Language Models Leaking Your Personal Information?cited by 200
- 2022Towards a Cleaner Document-Oriented Multilingual Crawled Corpuscited by 143
- 2020ParsBERT: Transformer-based Model for Persian Language Understandingcited by 93
- 2020LSCP: Enhanced Large Scale Colloquial Persian Language Understandingcited by 6
- 2021The Challenges of Persian User-generated Textual Content: A Machine Learning-Based Approachcited by 2
Built on by
Citing papers per year: 2025: 4; 2026: 1.
5 papers.
- 2025Matina: A Large-Scale 73B Token Persian Text Corpuscited by 3
- 2025MELAC: Massive Evaluation of Large Language Models with Alignment of Culture in Persian Languagecited by 3
- 2025PBBQ: A Persian Bias Benchmark Dataset Curated with Human-AI Collaboration for Large Language Modelscited by 1
- 2025PerHalluEval: Persian Hallucination Evaluation Benchmark for Large Language Modelscited by 0
- 2026Persian Pixel: A large-scale synthetic OCR dataset for Persian languagecited by 0
- 2026Persian Pixel: A large-scale synthetic OCR dataset for Persian language22 Jul 2026
- 2025PBBQ: A Persian Bias Benchmark Dataset Curated with Human-AI Collaboration for Large Language Models22 Oct 2025
- 2025PerHalluEval: Persian Hallucination Evaluation Benchmark for Large Language Models25 Sep 2025
- 2025MELAC: Massive Evaluation of Large Language Models with Alignment of Culture in Persian Language1 Aug 2025
- 2025Matina: A Large-Scale 73B Token Persian Text Corpus13 Feb 2025