# Arabic Dialect Identification under Scrutiny: Limitations of Single-label Classification

arXiv: https://arxiv.org/abs/2310.13661 · HTML: https://mlsanity.com/paper/2310.13661/

## Key facts

- Cited by 13 papers in the map (data through 14 Oct 2026).
- Builds on 9 of its 41 references that are in the map.
- First posted on arXiv 20 Oct 2023.

## Builds on (most cited first, 9 of 9)

- [ARBERT & MARBERT: Deep Bidirectional Transformers for Arabic](https://mlsanity.com/paper/2101.01785/) (2020)
- [NADI 2020: The First Nuanced Arabic Dialect Identification Shared Task](https://mlsanity.com/paper/2010.11334/) (2020)
- [Arabic Dialect Identification in the Wild](https://mlsanity.com/paper/2005.06557/) (2020)
- [NADI 2021: The Second Nuanced Arabic Dialect Identification Shared Task](https://mlsanity.com/paper/2103.08466/) (2021)
- [Arap-Tweet: A Large Multi-Dialect Twitter Corpus for Gender, Age and Language Variety Identification](https://mlsanity.com/paper/1808.07674/) (2018)
- [NADI 2022: The Third Nuanced Arabic Dialect Identification Shared Task](https://mlsanity.com/paper/2210.09582/) (2022)
- [Language Variety Identification with True Labels](https://mlsanity.com/paper/2303.01490/) (2023)
- [Automatic Arabic Dialect Identification Systems for Written Texts: A Survey](https://mlsanity.com/paper/2009.12622/) (2020)
- [FreCDo: A Large Corpus for French Cross-Domain Dialect Identification](https://mlsanity.com/paper/2212.07707/) (2022)

## Built on by (most cited first, 13 of 13)

- [NADI 2024: The Fifth Nuanced Arabic Dialect Identification Shared Task](https://mlsanity.com/paper/2407.04910/) (2024)
- [Swan and ArabicMTEB: Dialect-Aware, Arabic-Centric, Cross-Lingual, and Cross-Cultural Embedding Models and Benchmarks](https://mlsanity.com/paper/2411.01192/) (2024)
- [Exploiting Dialect Identification in Automatic Dialectal Text Normalization](https://mlsanity.com/paper/2407.03020/) (2024)
- [A Survey of Code-switched Arabic NLP: Progress, Challenges, and Future Directions](https://mlsanity.com/paper/2501.13419/) (2025)
- [Estimating the Level of Dialectness Predicts Interannotator Agreement in Multi-dialect Arabic Datasets](https://mlsanity.com/paper/2405.11282/) (2024)
- [Common Ground, Diverse Roots: The Difficulty of Classifying Common Examples in Spanish Varieties](https://mlsanity.com/paper/2412.11750/) (2024)
- [Multi-label Scandinavian Language Identification (SLIDE)](https://mlsanity.com/paper/2502.06692/) (2025)
- [Revisiting Common Assumptions about Arabic Dialects in NLP](https://mlsanity.com/paper/2505.21816/) (2025)
- [Moly'e: A Corpus-based Approach to Language Contact in Colonial France](https://mlsanity.com/paper/2408.04554/) (2024)
- [The Arabic Generality Score: Another Dimension of Modeling Arabic Dialectness](https://mlsanity.com/paper/2508.17347/) (2025)
- [How Should We Model the Probability of a Language?](https://mlsanity.com/paper/2602.08951/) (2026)
- [Curriculum Learning and Pseudo-Labeling Improve the Generalization of Multi-Label Arabic Dialect Identification Models](https://mlsanity.com/paper/2602.12937/) (2026)
- [Cultural Benchmarking of LLMs in Standard and Dialectal Arabic Dialogues](https://mlsanity.com/paper/2605.00119/) (2026)

## Abstract (arXiv)

Automatic Arabic Dialect Identification (ADI) of text has gained great popularity since it was introduced in the early 2010s. Multiple datasets were developed, and yearly shared tasks have been running since 2018. However, ADI systems are reported to fail in distinguishing between the micro-dialects of Arabic. We argue that the currently adopted framing of the ADI task as a single-label classification problem is one of the main reasons for that. We highlight the limitation of the incompleteness of the Dialect labels and demonstrate how it impacts the evaluation of ADI systems. A manual error analysis for the predictions of an ADI, performed by 7 native speakers of different Arabic dialects, revealed that ≈ 66% of the validated errors are not true errors. Consequently, we propose framing ADI as a multi-label classification task and give recommendations for designing new ADI datasets.