# Efficient Convolutional Neural Networks for Diacritic Restoration

arXiv: https://arxiv.org/abs/1912.06900 · HTML: https://mlsanity.com/paper/1912.06900/

## Key facts

- Cited by 9 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 14 Dec 2019.

## Built on by (most cited first, 9 of 9)

- [Automatic Restoration of Diacritics for Speech Data Sets](https://mlsanity.com/paper/2311.10771/) (2023)
- [Diacritics Restoration using BERT with Analysis on Czech language](https://mlsanity.com/paper/2105.11408/) (2021)
- [Improving Arabic Diacritization by Learning to Diacritize and Translate](https://mlsanity.com/paper/2109.14150/) (2021)
- [Correcting diacritics and typos with a ByT5 transformer model](https://mlsanity.com/paper/2201.13242/) (2022)
- [Bridging Gaps in Natural Language Processing for Yor`ub'a: A Systematic Review of a Decade of Progress and Prospects](https://mlsanity.com/paper/2502.17364/) (2025)
- [A Context-Contrastive Inference Approach To Partial Diacritization](https://mlsanity.com/paper/2401.08919/) (2024)
- [Interplay of Machine Translation, Diacritics, and Diacritization](https://mlsanity.com/paper/2404.05943/) (2024)
- [Proper Noun Diacritization for Arabic Wikipedia: A Benchmark Dataset](https://mlsanity.com/paper/2505.02656/) (2025)
- [Dilated Convolutional Neural Networks for Lightweight Diacritics Restoration](https://mlsanity.com/paper/2201.06757/) (2022)

## Abstract (arXiv)

Diacritic restoration has gained importance with the growing need for machines to understand written texts. The task is typically modeled as a sequence labeling problem and currently Bidirectional Long Short Term Memory (BiLSTM) models provide state-of-the-art results. Recently, Bai et al. (2018) show the advantages of Temporal Convolutional Neural Networks (TCN) over Recurrent Neural Networks (RNN) for sequence modeling in terms of performance and computational resources. As diacritic restoration benefits from both previous as well as subsequent timesteps, we further apply and evaluate a variant of TCN, Acausal TCN (A-TCN), which incorporates context from both directions (previous and future) rather than strictly incorporating previous context as in the case of TCN. A-TCN yields significant improvement over TCN for diacritization in three different languages: Arabic, Yoruba, and Vietnamese. Furthermore, A-TCN and BiLSTM have comparable performance, making A-TCN an efficient alternative over BiLSTM since convolutions can be trained in parallel. A-TCN is significantly faster than BiLSTM at inference time (270%-334% improvement in the amount of text diacritized per minute).