# Deep Diacritization: Efficient Hierarchical Recurrence for Improved Arabic Diacritization

arXiv: https://arxiv.org/abs/2011.00538 · HTML: https://mlsanity.com/paper/2011.00538/

## Key facts

- Cited by 11 papers in the map (data through 7 Oct 2026).
- First posted on arXiv 1 Nov 2020.

## Built on by (most cited first, 11 of 11)

- [Taqyim: Evaluating Arabic NLP Tasks Using ChatGPT Models](https://mlsanity.com/paper/2306.16322/) (2023)
- [Automatic Restoration of Diacritics for Speech Data Sets](https://mlsanity.com/paper/2311.10771/) (2023)
- [Arabic Diacritics in the Wild: Exploiting Opportunities for Improved Diacritization](https://mlsanity.com/paper/2406.05760/) (2024)
- [Diacritics Restoration using BERT with Analysis on Czech language](https://mlsanity.com/paper/2105.11408/) (2021)
- [AraReasoner: Evaluating Reasoning-Based LLMs for Arabic NLP](https://mlsanity.com/paper/2506.08768/) (2025)
- [Arabic Text Diacritization In The Age Of Transfer Learning: Token Classification Is All You Need](https://mlsanity.com/paper/2401.04848/) (2024)
- [CATT: Character-based Arabic Tashkeel Transformer](https://mlsanity.com/paper/2407.03236/) (2024)
- [Diacritic Recognition Performance in Arabic ASR](https://mlsanity.com/paper/2302.14022/) (2023)
- [Take the Hint: Improving Arabic Diacritization with Partially-Diacritized Text](https://mlsanity.com/paper/2306.03557/) (2023)
- [A Context-Contrastive Inference Approach To Partial Diacritization](https://mlsanity.com/paper/2401.08919/) (2024)
- [Sadeed: Advancing Arabic Diacritization Through Small Language Model](https://mlsanity.com/paper/2504.21635/) (2025)

## Abstract (arXiv)

We propose a novel architecture for labelling character sequences that achieves state-of-the-art results on the Tashkeela Arabic diacritization benchmark. The core is a two-level recurrence hierarchy that operates on the word and character levels separately---enabling faster training and inference than comparable traditional models. A cross-level attention module further connects the two, and opens the door for network interpretability. The task module is a softmax classifier that enumerates valid combinations of diacritics. This architecture can be extended with a recurrent decoder that optionally accepts priors from partially diacritized text, which improves results. We employ extra tricks such as sentence dropout and majority voting to further boost the final result. Our best model achieves a WER of 5.34%, outperforming the previous state-of-the-art with a 30.56% relative error reduction.