mlsanityData to 14 Oct 2026

Arabic Dialect Identification under Scrutiny: Limitations of Single-label Classification

arXiv 2310.13661 ↗First posted 20 Oct 2023cs.CLcited by 13

Not on the map. Why?

Abstract (arXiv)

Automatic Arabic Dialect Identification (ADI) of text has gained great popularity since it was introduced in the early 2010s. Multiple datasets were developed, and yearly shared tasks have been running since 2018. However, ADI systems are reported to fail in distinguishing between the micro-dialects of Arabic. We argue that the currently adopted framing of the ADI task as a single-label classification problem is one of the main reasons for that. We highlight the limitation of the incompleteness of the Dialect labels and demonstrate how it impacts the evaluation of ADI systems. A manual error analysis for the predictions of an ADI, performed by 7 native speakers of different Arabic dialects, revealed that ≈ 66% of the validated errors are not true errors. Consequently, we propose framing ADI as a multi-label classification task and give recommendations for designing new ADI datasets.

Key facts. Cited by 13 papers in the map (data through 14 Oct 2026). Builds on 9 of its 41 references that are in the map. First posted on arXiv 20 Oct 2023.

Builds on

9 of 41 references.

References · 9

  1. 2020ARBERT & MARBERT: Deep Bidirectional Transformers for Arabiccited by 166
  2. 2020NADI 2020: The First Nuanced Arabic Dialect Identification Shared Taskcited by 47
  3. 2020Arabic Dialect Identification in the Wildcited by 45
  4. 2021NADI 2021: The Second Nuanced Arabic Dialect Identification Shared Taskcited by 32
  5. 2018Arap-Tweet: A Large Multi-Dialect Twitter Corpus for Gender, Age and Language Variety Identificationcited by 23
  6. 2022NADI 2022: The Third Nuanced Arabic Dialect Identification Shared Taskcited by 23
  7. 2023Language Variety Identification with True Labelscited by 12
  8. 2020Automatic Arabic Dialect Identification Systems for Written Texts: A Surveycited by 10
  9. 2022FreCDo: A Large Corpus for French Cross-Domain Dialect Identificationcited by 1

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2024: 6; 2025: 4; 2026: 3.

13 papers.

Sort:

By area:

  1. 2024NADI 2024: The Fifth Nuanced Arabic Dialect Identification Shared Taskcited by 23
  2. 2024Swan and ArabicMTEB: Dialect-Aware, Arabic-Centric, Cross-Lingual, and Cross-Cultural Embedding Models and Benchmarkscited by 10
  3. 2024Exploiting Dialect Identification in Automatic Dialectal Text Normalizationcited by 7
  4. 2025A Survey of Code-switched Arabic NLP: Progress, Challenges, and Future Directionscited by 5
  5. 2024Estimating the Level of Dialectness Predicts Interannotator Agreement in Multi-dialect Arabic Datasetscited by 4
  6. 2024Common Ground, Diverse Roots: The Difficulty of Classifying Common Examples in Spanish Varietiescited by 3
  7. 2025Multi-label Scandinavian Language Identification (SLIDE)cited by 2
  8. 2025Revisiting Common Assumptions about Arabic Dialects in NLPcited by 2
  9. 2024Moly'e: A Corpus-based Approach to Language Contact in Colonial Francecited by 1
  10. 2025The Arabic Generality Score: Another Dimension of Modeling Arabic Dialectnesscited by 1
  11. 2026How Should We Model the Probability of a Language?cited by 0
  12. 2026Curriculum Learning and Pseudo-Labeling Improve the Generalization of Multi-Label Arabic Dialect Identification Modelscited by 0
1 more
  1. 2026Cultural Benchmarking of LLMs in Standard and Dialectal Arabic Dialoguescited by 0