# Improving Arabic Diacritization by Learning to Diacritize and Translate

arXiv: https://arxiv.org/abs/2109.14150 · HTML: https://mlsanity.com/paper/2109.14150/

## Key facts

- Cited by 6 papers in the map (data through 14 Oct 2026).
- Builds on 23 of its 45 references that are in the map.
- First posted on arXiv 29 Sep 2021.

## Builds on (most cited first, 23 of 23)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [Neural Machine Translation of Rare Words with Subword Units](https://mlsanity.com/paper/1508.07909/) (2015)
- [SentencePiece: A simple and language independent subword tokenizer and detokenizer for Neural Text Processing](https://mlsanity.com/paper/1808.06226/) (2018)
- [fairseq: A Fast, Extensible Toolkit for Sequence Modeling](https://mlsanity.com/paper/1904.01038/) (2019)
- [Mitigating Gender Bias in Natural Language Processing: Literature Review](https://mlsanity.com/paper/1906.08976/) (2019)
- [WikiMatrix: Mining 135M Parallel Sentences in 1620 Language Pairs from Wikipedia](https://mlsanity.com/paper/1907.05791/) (2019)
- [Transfer Learning for Low-Resource Neural Machine Translation](https://mlsanity.com/paper/1604.02201/) (2016)
- [CCAligned: A Massive Collection of Cross-Lingual Web-Document Pairs](https://mlsanity.com/paper/1911.06154/) (2019)
- [Automatic Machine Translation Evaluation in Many Languages via Zero-Shot Paraphrasing](https://mlsanity.com/paper/2004.14564/) (2020)
- [Rapid Adaptation of Neural Machine Translation to New Languages](https://mlsanity.com/paper/1808.04189/) (2018)
- [On the Impact of Various Types of Noise on Neural Machine Translation](https://mlsanity.com/paper/1805.12282/) (2018)
- [Transfer Learning across Low-Resource, Related Languages for Neural Machine Translation](https://mlsanity.com/paper/1708.09803/) (2017)
- [Fully Character-Level Neural Machine Translation without Explicit Segmentation](https://mlsanity.com/paper/1610.03017/) (2016)
- [Revisiting Character-Based Neural Machine Translation with Capacity and Compression](https://mlsanity.com/paper/1808.09943/) (2018)
- [ZEN 2.0: Continue Training and Adaption for N-gram Enhanced Text Encoders](https://mlsanity.com/paper/2105.01279/) (2021)
- [Arabic Text Diacritization Using Deep Neural Networks](https://mlsanity.com/paper/1905.01965/) (2019)
- [Joint Diacritization, Lemmatization, Normalization, and Fine-Grained Morphological Tagging](https://mlsanity.com/paper/1910.02267/) (2019)
- [A Multitask Learning Approach for Diacritic Restoration](https://mlsanity.com/paper/2006.04016/) (2020)
- [LDC Arabic Treebanks and Associated Corpora: Data Divisions Manual](https://mlsanity.com/paper/1309.5652/) (2013)
- [Adversarial Multitask Learning for Joint Multi-Feature and Multi-Dialect Morphological Modeling](https://mlsanity.com/paper/1910.12702/) (2019)
- [Efficient Convolutional Neural Networks for Diacritic Restoration](https://mlsanity.com/paper/1912.06900/) (2019)
- [Diacritics Restoration using BERT with Analysis on Czech language](https://mlsanity.com/paper/2105.11408/) (2021)
- [Homograph Disambiguation Through Selective Diacritic Restoration](https://mlsanity.com/paper/1912.04479/) (2019)

## Built on by (most cited first, 6 of 6)

- [Arabic Diacritics in the Wild: Exploiting Opportunities for Improved Diacritization](https://mlsanity.com/paper/2406.05760/) (2024)
- [Arabic Text Diacritization In The Age Of Transfer Learning: Token Classification Is All You Need](https://mlsanity.com/paper/2401.04848/) (2024)
- [Take the Hint: Improving Arabic Diacritization with Partially-Diacritized Text](https://mlsanity.com/paper/2306.03557/) (2023)
- [Multilingual context-based pronunciation learning for Text-to-Speech](https://mlsanity.com/paper/2307.16709/) (2023)
- [Interplay of Machine Translation, Diacritics, and Diacritization](https://mlsanity.com/paper/2404.05943/) (2024)
- [Acquiring Pronunciation Knowledge from Transcribed Speech Audio via Multi-task Learning](https://mlsanity.com/paper/2409.09891/) (2024)

## Abstract (arXiv)

We propose a novel multitask learning method for diacritization which trains a model to both diacritize and translate. Our method addresses data sparsity by exploiting large, readily available bitext corpora. Furthermore, translation requires implicit linguistic and semantic knowledge, which is helpful for resolving ambiguities in the diacritization task. We apply our method to the Penn Arabic Treebank and report a new state-of-the-art word error rate of 4.79%. We also conduct manual and automatic analysis to better understand our method and highlight some of the remaining challenges in diacritization.