# Exploiting Dialect Identification in Automatic Dialectal Text Normalization

arXiv: https://arxiv.org/abs/2407.03020 · HTML: https://mlsanity.com/paper/2407.03020/

## Key facts

- Cited by 7 papers in the map (data through 14 Oct 2026).
- Builds on 25 of its 80 references that are in the map.
- First posted on arXiv 3 Jul 2024.

## Builds on (most cited first, 25 of 25)

- [Neural Machine Translation of Rare Words with Subword Units](https://mlsanity.com/paper/1508.07909/) (2015)
- [HuggingFace's Transformers: State-of-the-art Natural Language Processing](https://mlsanity.com/paper/1910.03771/) (2019)
- [Google's Multilingual Neural Machine Translation System: Enabling Zero-Shot Translation](https://mlsanity.com/paper/1611.04558/) (2016)
- [GECToR -- Grammatical Error Correction: Tag, Not Rewrite](https://mlsanity.com/paper/2005.12592/) (2020)
- [AraT5: Text-to-Text Transformers for Arabic Language Generation](https://mlsanity.com/paper/2109.12068/) (2021)
- [Arabic Dialect Identification in the Wild](https://mlsanity.com/paper/2005.06557/) (2020)
- [Linguistic Input Features Improve Neural Machine Translation](https://mlsanity.com/paper/1606.02892/) (2016)
- [Controlling Text Complexity in Neural Machine Translation](https://mlsanity.com/paper/1911.00835/) (2019)
- [NADI 2021: The Second Nuanced Arabic Dialect Identification Shared Task](https://mlsanity.com/paper/2103.08466/) (2021)
- [NADI 2023: The Fourth Nuanced Arabic Dialect Identification Shared Task](https://mlsanity.com/paper/2310.16117/) (2023)
- [Multi-Task Neural Models for Translating Between Styles Within and Across Languages](https://mlsanity.com/paper/1806.04357/) (2018)
- [AraBART: a Pretrained Arabic Sequence-to-Sequence Model for Abstractive Summarization](https://mlsanity.com/paper/2203.10945/) (2022)
- [Arap-Tweet: A Large Multi-Dialect Twitter Corpus for Gender, Age and Language Variety Identification](https://mlsanity.com/paper/1808.07674/) (2018)
- [NADI 2022: The Third Nuanced Arabic Dialect Identification Shared Task](https://mlsanity.com/paper/2210.09582/) (2022)
- [Advancements in Arabic Grammatical Error Detection and Correction: An Empirical Investigation](https://mlsanity.com/paper/2305.14734/) (2023)
- [Automatic Error Type Annotation for Arabic](https://mlsanity.com/paper/2109.08068/) (2021)
- [A Large Scale Corpus of Gulf Arabic](https://mlsanity.com/paper/1609.02960/) (2016)
- [Controlling Pre-trained Language Models for Grade-Specific Text Simplification](https://mlsanity.com/paper/2305.14993/) (2023)
- [Arabic Dialect Identification under Scrutiny: Limitations of Single-label Classification](https://mlsanity.com/paper/2310.13661/) (2023)
- [Joint Diacritization, Lemmatization, Normalization, and Fine-Grained Morphological Tagging](https://mlsanity.com/paper/1910.02267/) (2019)
- [Beyond English: Evaluating LLMs for Arabic Grammatical Error Correction](https://mlsanity.com/paper/2312.08400/) (2023)
- [Camelira: An Arabic Multi-Dialect Morphological Disambiguator](https://mlsanity.com/paper/2211.16807/) (2022)
- [Utilizing Character and Word Embeddings for Text Normalization with Sequence-to-Sequence Models](https://mlsanity.com/paper/1809.01534/) (2018)
- [Octopus: A Multitask Model and Toolkit for Arabic Natural Language Generation](https://mlsanity.com/paper/2310.16127/) (2023)
- [Low Resourced Machine Translation via Morpho-syntactic Modeling: The Case of Dialectal Arabic](https://mlsanity.com/paper/1712.06273/) (2017)

## Built on by (most cited first, 7 of 7)

- [Enhancing Text Editing for Grammatical Error Correction: Arabic as a Case Study](https://mlsanity.com/paper/2503.00985/) (2025)
- [Advancing Dialectal Arabic to Modern Standard Arabic Machine Translation](https://mlsanity.com/paper/2507.20301/) (2025)
- [ADI-20: Arabic Dialect Identification dataset and models](https://mlsanity.com/paper/2511.10070/) (2025)
- [Ara-HOPE: Human-Centric Post-Editing Evaluation for Dialectal Arabic to Modern Standard Arabic Translation](https://mlsanity.com/paper/2512.21787/) (2025)
- [The Arabic Generality Score: Another Dimension of Modeling Arabic Dialectness](https://mlsanity.com/paper/2508.17347/) (2025)
- [ELYADATA & LIA at NADI 2025: ASR and ADI Subtasks](https://mlsanity.com/paper/2511.10090/) (2025)
- [BERnaT: Basque Encoders for Representing Natural Textual Diversity](https://mlsanity.com/paper/2512.03903/) (2025)

## Abstract (arXiv)

Dialectal Arabic is the primary spoken language used by native Arabic speakers in daily communication. The rise of social media platforms has notably expanded its use as a written language. However, Arabic dialects do not have standard orthographies. This, combined with the inherent noise in user-generated content on social media, presents a major challenge to NLP applications dealing with Dialectal Arabic. In this paper, we explore and report on the task of CODAfication, which aims to normalize Dialectal Arabic into the Conventional Orthography for Dialectal Arabic (CODA). We work with a unique parallel corpus of multiple Arabic dialects focusing on five major city dialects. We benchmark newly developed pretrained sequence-to-sequence models on the task of CODAfication. We further show that using dialect identification information improves the performance across all dialects. We make our code, data, and pretrained models publicly available.