The Annotation Bottleneck in Persian Text NLP: Persian as an Annotation-Scarce Language
Not on the map. Why?
Abstract (arXiv)
Persian (Farsi) is often described as a low-resource language in natural language processing, but that label collapses distinct shortages into a single category. This paper argues that Persian is more precisely described as annotation-scarce, provided that the term is understood as a property of its NLP resource ecology rather than an intrinsic property of the language. The review covers 34 representative Persian text resources available by July 2026 and adds three quantitative cross-checks. First, independent web measurements place Persian among roughly the twenty most visible content languages: W3Techs reports Persian on about 0.9% of websites with a known content language, while Common Crawl CC-MAIN-2026-30 identifies Persian as the primary language of 0.7039% of HTML pages. Second, a selective speech review shows a long resource trajectory from FARSDAT to recent corpora containing hundreds or thousands of hours of speech. Third, a matched Persian-English comparison normalizes task-specific annotation volumes by relative Common Crawl web presence. The resulting ratios vary sharply: Persian syntax and news NER are comparatively dense, whereas natural-language inference falls below the web-proportional baseline. The evidence therefore does not support a simple claim that Persian is globally deficient in labeled volume. Instead, annotation scarcity is expressed through uneven task and domain coverage, incompatible schemes, access and documentation friction, and limited supervision for specialist domains, preference data, and varieties beyond standard Iranian Persian.
Key facts. No citing papers in the map yet (data through 7 Oct 2026). Builds on 32 of its 58 references that are in the map. First posted on arXiv 25 Aug 2026.
Builds on
32 of 58 references.
References · 32
- 2017A Broad-Coverage Challenge Corpus for Sentence Understanding through Inferencecited by 2,665
- 2015A large annotated corpus for learning natural language inferencecited by 1,837
- 2018Datasheets for Datasetscited by 1,371
- 2017Get To The Point: Summarization with Pointer-Generator Networkscited by 1,281
- 2020The State and Fate of Linguistic Diversity and Inclusion in the NLP Worldcited by 789
- 2024Aya Model: An Instruction Finetuned Open-Access Multilingual Language Modelcited by 292
- 2020A Survey on Recent Approaches for Natural Language Processing in Low-Resource Scenarioscited by 170
- 2020ParsBERT: Transformer-based Model for Persian Language Understandingcited by 92
- 2020ParsiNLU: A Suite of Language Understanding Challenges for Persiancited by 42
- 2020FarsTail: A Persian Natural Language Inference Datasetcited by 23
- 2024Khayyam Challenge (PersianMMLU): Is Your LLM Truly Wise to The Persian Language?cited by 22
- 2018SentiPers: A Sentiment Analysis Corpus for Persiancited by 14
20 more
- 2018MIZAN: A Large Persian-English Parallel Corpuscited by 13
- 2020Leveraging ParsBERT and Pretrained mT5 for Persian Abstractive Text Summarizationcited by 11
- 2024Benchmarking Large Language Models for Persian: A Preliminary Study Focusing on ChatGPTcited by 11
- 2019A Multi Purpose and Large Scale Speech Corpus in Persian and English for Speaker and Speech Recognition: the DeepMine Databasecited by 10
- 2022ArmanEmo: A Persian Dataset for Text-based Emotion Detectioncited by 9
- 2024PersianMind: A Cross-Lingual Persian-English Large Language Modelcited by 7
- 2021Persian Rhetorical Structure Theorycited by 6
- 2022Graphemic Normalization of the Perso-Arabic Scriptcited by 6
- 2018PEYMA: A Tagged Corpus for Persian Named Entitiescited by 5
- 2021HmBlogs: A big general Persian corpuscited by 5
- 2021PerCQA: Persian Community Question Answering Datasetcited by 5
- 2023A Multi-Purpose Audio-Visual Corpus for Multi-Modal Persian Speech Recognition: the Arman-AV Datasetcited by 4
- 2025Matina: A Large-Scale 73B Token Persian Text Corpuscited by 3
- 2025ParsVoice: A Large-Scale Multi-Speaker Persian Speech Corpus for Text-to-Speech Synthesiscited by 3
- 2020NSURL-2019 Task 7: Named Entity Recognition (NER) in Farsicited by 2
- 2022A hybrid entity-centric approach to Persian pronoun resolutioncited by 2
- 2025FarsEval-PKBETS: A new diverse benchmark for evaluating Persian large language modelscited by 2
- 2025Benchmarking Open-Source Large Language Models for Persian in Zero-Shot and Few-Shot Learningcited by 1
- 2026PARSE: An Open-Domain Reasoning Question Answering Benchmark for Persiancited by 1
- 2026PersianPunc: A Large-Scale Dataset and BERT-Based Approach for Persian Punctuation Restorationcited by 1
Built on by
Not cited yet.