mlsanityData to 7 Oct 2026

Arabic Text Diacritization In The Age Of Transfer Learning: Token Classification Is All You Need

arXiv 2401.04848 ↗First posted 9 Jan 2024cs.CLcited by 4

Not on the map. Why?

Abstract (arXiv)

Automatic diacritization of Arabic text involves adding diacritical marks (diacritics) to the text. This task poses a significant challenge with noteworthy implications for computational processing and comprehension. In this paper, we introduce PTCAD (Pre-FineTuned Token Classification for Arabic Diacritization, a novel two-phase approach for the Arabic Text Diacritization task. PTCAD comprises a pre-finetuning phase and a finetuning phase, treating Arabic Text Diacritization as a token classification task for pre-trained models. The effectiveness of PTCAD is demonstrated through evaluations on two benchmark datasets derived from the Tashkeela dataset, where it achieves state-of-the-art results, including a 20% reduction in Word Error Rate (WER) compared to existing benchmarks and superior performance over GPT-4 in ATD tasks.

Key facts. Cited by 4 papers in the map (data through 7 Oct 2026). Builds on 20 of its 50 references that are in the map. First posted on arXiv 9 Jan 2024.

Builds on

20 of 50 references.

References · 20

  1. 2017Attention Is All You Needcited by 59,932
  2. 2019Sentence-BERT: Sentence Embeddings using Siamese BERT-Networkscited by 9,049
  3. 2020Longformer: The Long-Document Transformercited by 3,362
  4. 2020Big Bird: Transformers for Longer Sequencescited by 1,673
  5. 2020AraBERT: Transformer-based Model for Arabic Language Understandingcited by 359
  6. 2020ARBERT & MARBERT: Deep Bidirectional Transformers for Arabiccited by 164
  7. 2021The Interplay of Variant, Size, and Task Type in Arabic Pre-trained Language Modelscited by 118
  8. 2019Arabic Text Diacritization Using Deep Neural Networkscited by 18
  9. 2019Learning meters of Arabic and English poems with Recurrent Neural Networks: a step forward for language understanding and synthesiscited by 13
  10. 2019Joint Diacritization, Lemmatization, Normalization, and Fine-Grained Morphological Taggingcited by 12
  11. 2020A Multitask Learning Approach for Diacritic Restorationcited by 12
  12. 2021Deep Multi-Task Model for Sarcasm Detection and Sentiment Analysis in Arabic Languagecited by 12
8 more
  1. 2013LDC Arabic Treebanks and Associated Corpora: Data Divisions Manualcited by 11
  2. 2019Neural Arabic Text Diacritization: State of the Art Results and a Novel Approach for Machine Translationcited by 11
  3. 2020Deep Diacritization: Efficient Hierarchical Recurrence for Improved Arabic Diacritizationcited by 11
  4. 2018Studying the History of the Arabic Language: Language Technology and a Large-Scale Historical Corpuscited by 7
  5. 2020Arabic Diacritic Recovery Using a Feature-Rich biLSTM Modelcited by 7
  6. 2021BERT-based Multi-Task Model for Country and Province Level Modern Standard Arabic and Dialectal Arabic Identificationcited by 7
  7. 2021Improving Arabic Diacritization by Learning to Diacritize and Translatecited by 6
  8. 2022Natural Language Processing in Customer Service: A Systematic Reviewcited by 4

Built on by

Cited by year

Citing papers per year: 2024: 2; 2025: 2.

4 papers.

  1. 2024CATT: Character-based Arabic Tashkeel Transformercited by 4
  2. 2024YAD: Leveraging T5 for Improved Automatic Diacritization of Yor`ub'a Textcited by 4
  3. 2025Are LLMs Good Text Diacritizers? An Arabic and Yoruba Case Studycited by 2
  4. 2025Sadeed: Advancing Arabic Diacritization Through Small Language Modelcited by 0