# Arabic Text Diacritization In The Age Of Transfer Learning: Token Classification Is All You Need

arXiv: https://arxiv.org/abs/2401.04848 · HTML: https://mlsanity.com/paper/2401.04848/

## Key facts

- Cited by 4 papers in the map (data through 14 Oct 2026).
- Builds on 20 of its 50 references that are in the map.
- First posted on arXiv 9 Jan 2024.

## Builds on (most cited first, 20 of 20)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [Sentence-BERT: Sentence Embeddings using Siamese BERT-Networks](https://mlsanity.com/paper/1908.10084/) (2019)
- [Longformer: The Long-Document Transformer](https://mlsanity.com/paper/2004.05150/) (2020)
- [Big Bird: Transformers for Longer Sequences](https://mlsanity.com/paper/2007.14062/) (2020)
- [AraBERT: Transformer-based Model for Arabic Language Understanding](https://mlsanity.com/paper/2003.00104/) (2020)
- [ARBERT & MARBERT: Deep Bidirectional Transformers for Arabic](https://mlsanity.com/paper/2101.01785/) (2020)
- [The Interplay of Variant, Size, and Task Type in Arabic Pre-trained Language Models](https://mlsanity.com/paper/2103.06678/) (2021)
- [Arabic Text Diacritization Using Deep Neural Networks](https://mlsanity.com/paper/1905.01965/) (2019)
- [Learning meters of Arabic and English poems with Recurrent Neural Networks: a step forward for language understanding and synthesis](https://mlsanity.com/paper/1905.05700/) (2019)
- [Joint Diacritization, Lemmatization, Normalization, and Fine-Grained Morphological Tagging](https://mlsanity.com/paper/1910.02267/) (2019)
- [A Multitask Learning Approach for Diacritic Restoration](https://mlsanity.com/paper/2006.04016/) (2020)
- [Deep Multi-Task Model for Sarcasm Detection and Sentiment Analysis in Arabic Language](https://mlsanity.com/paper/2106.12488/) (2021)
- [LDC Arabic Treebanks and Associated Corpora: Data Divisions Manual](https://mlsanity.com/paper/1309.5652/) (2013)
- [Neural Arabic Text Diacritization: State of the Art Results and a Novel Approach for Machine Translation](https://mlsanity.com/paper/1911.03531/) (2019)
- [Deep Diacritization: Efficient Hierarchical Recurrence for Improved Arabic Diacritization](https://mlsanity.com/paper/2011.00538/) (2020)
- [Studying the History of the Arabic Language: Language Technology and a Large-Scale Historical Corpus](https://mlsanity.com/paper/1809.03891/) (2018)
- [Arabic Diacritic Recovery Using a Feature-Rich biLSTM Model](https://mlsanity.com/paper/2002.01207/) (2020)
- [BERT-based Multi-Task Model for Country and Province Level Modern Standard Arabic and Dialectal Arabic Identification](https://mlsanity.com/paper/2106.12495/) (2021)
- [Improving Arabic Diacritization by Learning to Diacritize and Translate](https://mlsanity.com/paper/2109.14150/) (2021)
- [Natural Language Processing in Customer Service: A Systematic Review](https://mlsanity.com/paper/2212.09523/) (2022)

## Built on by (most cited first, 4 of 4)

- [CATT: Character-based Arabic Tashkeel Transformer](https://mlsanity.com/paper/2407.03236/) (2024)
- [YAD: Leveraging T5 for Improved Automatic Diacritization of Yor`ub'a Text](https://mlsanity.com/paper/2412.20218/) (2024)
- [Are LLMs Good Text Diacritizers? An Arabic and Yoruba Case Study](https://mlsanity.com/paper/2506.11602/) (2025)
- [Sadeed: Advancing Arabic Diacritization Through Small Language Model](https://mlsanity.com/paper/2504.21635/) (2025)

## Abstract (arXiv)

Automatic diacritization of Arabic text involves adding diacritical marks (diacritics) to the text. This task poses a significant challenge with noteworthy implications for computational processing and comprehension. In this paper, we introduce PTCAD (Pre-FineTuned Token Classification for Arabic Diacritization, a novel two-phase approach for the Arabic Text Diacritization task. PTCAD comprises a pre-finetuning phase and a finetuning phase, treating Arabic Text Diacritization as a token classification task for pre-trained models. The effectiveness of PTCAD is demonstrated through evaluations on two benchmark datasets derived from the Tashkeela dataset, where it achieves state-of-the-art results, including a 20% reduction in Word Error Rate (WER) compared to existing benchmarks and superior performance over GPT-4 in ATD tasks.