mlsanityData to 7 Oct 2026

Exploiting Dialect Identification in Automatic Dialectal Text Normalization

arXiv 2407.03020 ↗First posted 3 Jul 2024cs.CLcited by 7

Not on the map. Why?

Abstract (arXiv)

Dialectal Arabic is the primary spoken language used by native Arabic speakers in daily communication. The rise of social media platforms has notably expanded its use as a written language. However, Arabic dialects do not have standard orthographies. This, combined with the inherent noise in user-generated content on social media, presents a major challenge to NLP applications dealing with Dialectal Arabic. In this paper, we explore and report on the task of CODAfication, which aims to normalize Dialectal Arabic into the Conventional Orthography for Dialectal Arabic (CODA). We work with a unique parallel corpus of multiple Arabic dialects focusing on five major city dialects. We benchmark newly developed pretrained sequence-to-sequence models on the task of CODAfication. We further show that using dialect identification information improves the performance across all dialects. We make our code, data, and pretrained models publicly available.

Key facts. Cited by 7 papers in the map (data through 7 Oct 2026). Builds on 25 of its 80 references that are in the map. First posted on arXiv 3 Jul 2024.

Builds on

25 of 80 references.

References · 25

  1. 2015Neural Machine Translation of Rare Words with Subword Unitscited by 2,898
  2. 2019HuggingFace's Transformers: State-of-the-art Natural Language Processingcited by 2,201
  3. 2016Google's Multilingual Neural Machine Translation System: Enabling Zero-Shot Translationcited by 467
  4. 2020GECToR -- Grammatical Error Correction: Tag, Not Rewritecited by 177
  5. 2021AraT5: Text-to-Text Transformers for Arabic Language Generationcited by 63
  6. 2020Arabic Dialect Identification in the Wildcited by 45
  7. 2016Linguistic Input Features Improve Neural Machine Translationcited by 38
  8. 2019Controlling Text Complexity in Neural Machine Translationcited by 32
  9. 2021NADI 2021: The Second Nuanced Arabic Dialect Identification Shared Taskcited by 32
  10. 2023NADI 2023: The Fourth Nuanced Arabic Dialect Identification Shared Taskcited by 29
  11. 2018Multi-Task Neural Models for Translating Between Styles Within and Across Languagescited by 28
  12. 2022AraBART: a Pretrained Arabic Sequence-to-Sequence Model for Abstractive Summarizationcited by 24
13 more
  1. 2018Arap-Tweet: A Large Multi-Dialect Twitter Corpus for Gender, Age and Language Variety Identificationcited by 23
  2. 2022NADI 2022: The Third Nuanced Arabic Dialect Identification Shared Taskcited by 23
  3. 2023Advancements in Arabic Grammatical Error Detection and Correction: An Empirical Investigationcited by 22
  4. 2021Automatic Error Type Annotation for Arabiccited by 21
  5. 2016A Large Scale Corpus of Gulf Arabiccited by 17
  6. 2023Controlling Pre-trained Language Models for Grade-Specific Text Simplificationcited by 14
  7. 2023Arabic Dialect Identification under Scrutiny: Limitations of Single-label Classificationcited by 13
  8. 2019Joint Diacritization, Lemmatization, Normalization, and Fine-Grained Morphological Taggingcited by 12
  9. 2023Beyond English: Evaluating LLMs for Arabic Grammatical Error Correctioncited by 12
  10. 2022Camelira: An Arabic Multi-Dialect Morphological Disambiguatorcited by 10
  11. 2018Utilizing Character and Word Embeddings for Text Normalization with Sequence-to-Sequence Modelscited by 7
  12. 2023Octopus: A Multitask Model and Toolkit for Arabic Natural Language Generationcited by 6
  13. 2017Low Resourced Machine Translation via Morpho-syntactic Modeling: The Case of Dialectal Arabiccited by 3

Built on by

Cited by year

Citing papers per year: 2025: 7.

7 papers.

  1. 2025Enhancing Text Editing for Grammatical Error Correction: Arabic as a Case Studycited by 9
  2. 2025Advancing Dialectal Arabic to Modern Standard Arabic Machine Translationcited by 6
  3. 2025ADI-20: Arabic Dialect Identification dataset and modelscited by 3
  4. 2025Ara-HOPE: Human-Centric Post-Editing Evaluation for Dialectal Arabic to Modern Standard Arabic Translationcited by 2
  5. 2025The Arabic Generality Score: Another Dimension of Modeling Arabic Dialectnesscited by 1
  6. 2025ELYADATA & LIA at NADI 2025: ASR and ADI Subtaskscited by 1
  7. 2025BERnaT: Basque Encoders for Representing Natural Textual Diversitycited by 0