# Take the Hint: Improving Arabic Diacritization with Partially-Diacritized Text

arXiv: https://arxiv.org/abs/2306.03557 · HTML: https://mlsanity.com/paper/2306.03557/

## Key facts

- Cited by 2 papers in the map (data through 14 Oct 2026).
- Builds on 16 of its 37 references that are in the map.
- First posted on arXiv 6 Jun 2023.

## Builds on (most cited first, 16 of 16)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [Neural Machine Translation by Jointly Learning to Align and Translate](https://mlsanity.com/paper/1409.0473/) (2014)
- [Sequence to Sequence Learning with Neural Networks](https://mlsanity.com/paper/1409.3215/) (2014)
- [RETURNN as a Generic Flexible Neural Toolkit with Application to Translation and Speech Recognition](https://mlsanity.com/paper/1805.05225/) (2018)
- [ZEN 2.0: Continue Training and Adaption for N-gram Enhanced Text Encoders](https://mlsanity.com/paper/2105.01279/) (2021)
- [Arabic Text Diacritization Using Deep Neural Networks](https://mlsanity.com/paper/1905.01965/) (2019)
- [Joint Diacritization, Lemmatization, Normalization, and Fine-Grained Morphological Tagging](https://mlsanity.com/paper/1910.02267/) (2019)
- [A Multitask Learning Approach for Diacritic Restoration](https://mlsanity.com/paper/2006.04016/) (2020)
- [LDC Arabic Treebanks and Associated Corpora: Data Divisions Manual](https://mlsanity.com/paper/1309.5652/) (2013)
- [Neural Arabic Text Diacritization: State of the Art Results and a Novel Approach for Machine Translation](https://mlsanity.com/paper/1911.03531/) (2019)
- [Deep Diacritization: Efficient Hierarchical Recurrence for Improved Arabic Diacritization](https://mlsanity.com/paper/2011.00538/) (2020)
- [Adversarial Multitask Learning for Joint Multi-Feature and Multi-Dialect Morphological Modeling](https://mlsanity.com/paper/1910.12702/) (2019)
- [Arabic Diacritic Recovery Using a Feature-Rich biLSTM Model](https://mlsanity.com/paper/2002.01207/) (2020)
- [Improving Arabic Diacritization by Learning to Diacritize and Translate](https://mlsanity.com/paper/2109.14150/) (2021)

## Built on by (most cited first, 2 of 2)

- [Arabic Diacritics in the Wild: Exploiting Opportunities for Improved Diacritization](https://mlsanity.com/paper/2406.05760/) (2024)
- [Sadeed: Advancing Arabic Diacritization Through Small Language Model](https://mlsanity.com/paper/2504.21635/) (2025)

## Abstract (arXiv)

Automatic Arabic diacritization is useful in many applications, ranging from reading support for language learners to accurate pronunciation predictor for downstream tasks like speech synthesis. While most of the previous works focused on models that operate on raw non-diacritized text, production systems can gain accuracy by first letting humans partly annotate ambiguous words. In this paper, we propose 2SDiac, a multi-source model that can effectively support optional diacritics in input to inform all predictions. We also introduce Guided Learning, a training scheme to leverage given diacritics in input with different levels of random masking. We show that the provided hints during test affect more output positions than those annotated. Moreover, experiments on two common benchmarks show that our approach i) greatly outperforms the baseline also when evaluated on non-diacritized text; and ii) achieves state-of-the-art results while reducing the parameter count by over 60%.