# ALDi: Quantifying the Arabic Level of Dialectness of Text arXiv: https://arxiv.org/abs/2310.13747 ยท HTML: https://mlsanity.com/paper/2310.13747/ ## Key facts - Cited by 24 papers in the map (data through 14 Oct 2026). - Builds on 13 of its 56 references that are in the map. - First posted on arXiv 20 Oct 2023. ## Builds on (most cited first, 13 of 13) - [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018) - [Dear Sir or Madam, May I introduce the GYAFC Dataset: Corpus, Benchmarks and Metrics for Formality Style Transfer](https://mlsanity.com/paper/1803.06535/) (2018) - [ARBERT & MARBERT: Deep Bidirectional Transformers for Arabic](https://mlsanity.com/paper/2101.01785/) (2020) - [NADI 2020: The First Nuanced Arabic Dialect Identification Shared Task](https://mlsanity.com/paper/2010.11334/) (2020) - [Arabic Dialect Identification in the Wild](https://mlsanity.com/paper/2005.06557/) (2020) - [Overview for the Second Shared Task on Language Identification in Code-Switched Data](https://mlsanity.com/paper/1909.13016/) (2019) - [NADI 2021: The Second Nuanced Arabic Dialect Identification Shared Task](https://mlsanity.com/paper/2103.08466/) (2021) - [Learning to Recognize Dialect Features](https://mlsanity.com/paper/2010.12707/) (2020) - [Arap-Tweet: A Large Multi-Dialect Twitter Corpus for Gender, Age and Language Variety Identification](https://mlsanity.com/paper/1808.07674/) (2018) - [NADI 2022: The Third Nuanced Arabic Dialect Identification Shared Task](https://mlsanity.com/paper/2210.09582/) (2022) - [Towards Responsible Natural Language Annotation for the Varieties of Arabic](https://mlsanity.com/paper/2203.09597/) (2022) - [SQUINKY! A Corpus of Sentence-level Formality, Informativeness, and Implicature](https://mlsanity.com/paper/1506.02306/) (2015) - [Automatic Arabic Dialect Identification Systems for Written Texts: A Survey](https://mlsanity.com/paper/2009.12622/) (2020) ## Built on by (most cited first, 24 of 24) - [Natural Language Processing for Dialects of a Language: A Survey](https://mlsanity.com/paper/2401.05632/) (2024) - [NADI 2024: The Fifth Nuanced Arabic Dialect Identification Shared Task](https://mlsanity.com/paper/2407.04910/) (2024) - [NileChat: Towards Linguistically Diverse and Culturally Aware LLMs for Local Communities](https://mlsanity.com/paper/2505.18383/) (2025) - [AL-QASIDA: Analyzing LLM Quality and Accuracy Systematically in Dialectal Arabic](https://mlsanity.com/paper/2412.04193/) (2024) - [JEEM: Vision-Language Understanding in Four Arabic Dialects](https://mlsanity.com/paper/2503.21910/) (2025) - [CommonLID: Re-evaluating State-of-the-Art Language Identification Performance on Web Data](https://mlsanity.com/paper/2601.18026/) (2026) - [A Survey of Code-switched Arabic NLP: Progress, Challenges, and Future Directions](https://mlsanity.com/paper/2501.13419/) (2025) - [Estimating the Level of Dialectness Predicts Interannotator Agreement in Multi-dialect Arabic Datasets](https://mlsanity.com/paper/2405.11282/) (2024) - [Arab Voices: Mapping Standard and Dialectal Arabic Speech Technology](https://mlsanity.com/paper/2601.13319/) (2026) - [Konooz: Multi-domain Multi-dialect Corpus for Named Entity Recognition](https://mlsanity.com/paper/2506.12615/) (2025) - [ArabJobs: A Multinational Corpus of Arabic Job Ads](https://mlsanity.com/paper/2509.22589/) (2025) - [Revisiting Common Assumptions about Arabic Dialects in NLP](https://mlsanity.com/paper/2505.21816/) (2025) - [Arabic Automatic Story Generation with Large Language Models](https://mlsanity.com/paper/2407.07551/) (2024) - [DialUp! Modeling the Language Continuum by Adapting Models to Dialects and Dialects to Models](https://mlsanity.com/paper/2501.16581/) (2025) - [The Arabic Generality Score: Another Dimension of Modeling Arabic Dialectness](https://mlsanity.com/paper/2508.17347/) (2025) - [From FusHa to Folk: Exploring Cross-Lingual Transfer in Arabic Language Models](https://mlsanity.com/paper/2602.09826/) (2026) - [Learning the Arabic Dialect Continuum as a Continuous Space: A Regression Approach to Speaker Origin Prediction](https://mlsanity.com/paper/2607.19751/) (2026) - [Voice Conversion Improves Cross-Domain Robustness for Spoken Arabic Dialect Identification](https://mlsanity.com/paper/2505.24713/) (2025) - [Maastricht University at AMIYA: Adapting LLMs for Dialectal Arabic using Fine-tuning and MBR Decoding](https://mlsanity.com/paper/2602.09703/) (2026) - [Curriculum Learning and Pseudo-Labeling Improve the Generalization of Multi-Label Arabic Dialect Identification Models](https://mlsanity.com/paper/2602.12937/) (2026) - [Context-Aware Dialectal Arabic Machine Translation with Interactive Region and Register Selection](https://mlsanity.com/paper/2604.06456/) (2026) - [Cultural Benchmarking of LLMs in Standard and Dialectal Arabic Dialogues](https://mlsanity.com/paper/2605.00119/) (2026) - [Romanized Arabic Across Dialects: Views, Usage Patterns, and Linguistic Variation](https://mlsanity.com/paper/2608.02555/) (2026) - [CARDAMOM: A Micro-Dialectal Arabic Speech Dataset for ASR](https://mlsanity.com/paper/2609.34481/) (2026) ## Abstract (arXiv) Transcribed speech and user-generated text in Arabic typically contain a mixture of Modern Standard Arabic (MSA), the standardized language taught in schools, and Dialectal Arabic (DA), used in daily communications. To handle this variation, previous work in Arabic NLP has focused on Dialect Identification (DI) on the sentence or the token level. However, DI treats the task as binary, whereas we argue that Arabic speakers perceive a spectrum of dialectness, which we operationalize at the sentence level as the Arabic Level of Dialectness (ALDi), a continuous linguistic variable. We introduce the AOC-ALDi dataset (derived from the AOC dataset), containing 127,835 sentences (17% from news articles and 83% from user comments on those articles) which are manually labeled with their level of dialectness. We provide a detailed analysis of AOC-ALDi and show that a model trained on it can effectively identify levels of dialectness on a range of other corpora (including dialects and genres not included in AOC-ALDi), providing a more nuanced picture than traditional DI systems. Through case studies, we illustrate how ALDi can reveal Arabic speakers' stylistic choices in different situations, a useful property for sociolinguistic analyses.