# Automatic Restoration of Diacritics for Speech Data Sets

arXiv: https://arxiv.org/abs/2311.10771 · HTML: https://mlsanity.com/paper/2311.10771/

## Key facts

- Cited by 8 papers in the map (data through 14 Oct 2026).
- Builds on 12 of its 26 references that are in the map.
- First posted on arXiv 15 Nov 2023.

## Builds on (most cited first, 12 of 12)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [Robust Speech Recognition via Large-Scale Weak Supervision](https://mlsanity.com/paper/2212.04356/) (2022)
- [Longformer: The Long-Document Transformer](https://mlsanity.com/paper/2004.05150/) (2020)
- [The Impact of Positional Encoding on Length Generalization in Transformers](https://mlsanity.com/paper/2305.19466/) (2023)
- [QASR: QCRI Aljazeera Speech Resource -- A Large Scale Annotated Arabic Speech Corpus](https://mlsanity.com/paper/2106.13000/) (2021)
- [Arabic Text Diacritization Using Deep Neural Networks](https://mlsanity.com/paper/1905.01965/) (2019)
- [ClArTTS: An Open-Source Classical Arabic Text-to-Speech Corpus](https://mlsanity.com/paper/2303.00069/) (2023)
- [Neural Arabic Text Diacritization: State of the Art Results and a Novel Approach for Machine Translation](https://mlsanity.com/paper/1911.03531/) (2019)
- [Deep Diacritization: Efficient Hierarchical Recurrence for Improved Arabic Diacritization](https://mlsanity.com/paper/2011.00538/) (2020)
- [Camelira: An Arabic Multi-Dialect Morphological Disambiguator](https://mlsanity.com/paper/2211.16807/) (2022)
- [Efficient Convolutional Neural Networks for Diacritic Restoration](https://mlsanity.com/paper/1912.06900/) (2019)
- [Unsupervised Data Selection for TTS: Using Arabic Broadcast News as a Case Study](https://mlsanity.com/paper/2301.09099/) (2023)

## Built on by (most cited first, 8 of 8)

- [NADI 2025: The First Multidialectal Arabic Speech Processing Shared Task](https://mlsanity.com/paper/2509.02038/) (2025)
- [ArVoice: A Multi-Speaker Dataset for Arabic Speech Synthesis](https://mlsanity.com/paper/2505.20506/) (2025)
- [Abjad AI at NADI 2025: CATT-Whisper: Multimodal Diacritic Restoration Using Text and Speech Representations](https://mlsanity.com/paper/2510.24247/) (2025)
- [Are LLMs Good Text Diacritizers? An Arabic and Yoruba Case Study](https://mlsanity.com/paper/2506.11602/) (2025)
- [ReNikud: Audio-Supervised Hebrew Grapheme-to-Phoneme Conversion](https://mlsanity.com/paper/2606.20179/) (2026)
- [Hebrew Diacritics Restoration using Visual Representation](https://mlsanity.com/paper/2510.26521/) (2025)
- [Constrained CTC Decoding for Efficient Diacritic Restoration](https://mlsanity.com/paper/2607.18946/) (2026)
- [SHAMS: An Audio-Grounded Pronunciation Benchmark for Levantine Arabic](https://mlsanity.com/paper/2610.01427/) (2026)

## Abstract (arXiv)

Automatic text-based diacritic restoration models generally have high diacritic error rates when applied to speech transcripts as a result of domain and style shifts in spoken language. In this work, we explore the possibility of improving the performance of automatic diacritic restoration when applied to speech data by utilizing parallel spoken utterances. In particular, we use the pre-trained Whisper ASR model fine-tuned on relatively small amounts of diacritized Arabic speech data to produce rough diacritized transcripts for the speech utterances, which we then use as an additional input for diacritic restoration models. The proposed framework consistently improves diacritic restoration performance compared to text-only baselines. Our results highlight the inadequacy of current text-based diacritic restoration models for speech data sets and provide a new baseline for speech-based diacritic restoration.