# Yo-ByT5: Efficient and High-Fidelity Diacritic Restoration for Yor`ub'a

arXiv: https://arxiv.org/abs/2610.01634 · HTML: https://mlsanity.com/paper/2610.01634/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 9 of its 21 references that are in the map.
- First posted on arXiv 1 Oct 2026.

## Builds on (most cited first, 9 of 9)

- [mT5: A massively multilingual pre-trained text-to-text transformer](https://mlsanity.com/paper/2010.11934/) (2020)
- [A Call for Clarity in Reporting BLEU Scores](https://mlsanity.com/paper/1804.08771/) (2018)
- [ByT5: Towards a token-free future with pre-trained byte-to-byte models](https://mlsanity.com/paper/2105.13626/) (2021)
- [The Effect of Domain and Diacritics in Yor`ub'a-English Neural Machine Translation](https://mlsanity.com/paper/2103.08647/) (2021)
- [Arabic Text Diacritization Using Deep Neural Networks](https://mlsanity.com/paper/1905.01965/) (2019)
- [Attentive Sequence-to-Sequence Learning for Diacritic Restoration of Yor`ub'a Language Text](https://mlsanity.com/paper/1804.00832/) (2018)
- [YAD: Leveraging T5 for Improved Automatic Diacritization of Yor`ub'a Text](https://mlsanity.com/paper/2412.20218/) (2024)
- [Bridging Gaps in Natural Language Processing for Yor`ub'a: A Systematic Review of a Decade of Progress and Prospects](https://mlsanity.com/paper/2502.17364/) (2025)
- [Are LLMs Good Text Diacritizers? An Arabic and Yoruba Case Study](https://mlsanity.com/paper/2506.11602/) (2025)

## Abstract (arXiv)

Yor`ub'a is a widely spoken tonal language that depends on diacritics to avoid lexical ambiguity. However, it is often written without these diacritics, thereby hindering downstream Natural Language Processing (NLP) tasks. In this paper, we introduce Yo-ByT5, a byte-level Automatic Diacritic Restoration (ADR) model fine-tuned from ByT5-small. We evaluate Yo-ByT5 alongside five publicly released Yor`ub'a ADR models and one open-weight large language model (LLM) on the YAD benchmark under a consistent protocol. Our results demonstrate that Yo-ByT5 matches the performance of the strongest existing model, mT5-base, with a DER of 10.14% and a CER of 3.48%. Furthermore, it exhibits superior text fidelity despite using approximately half the parameter count of mT5-base. We also release our training code and model outputs, as well as call for the development of a larger, purpose-built benchmark for Yor`ub'a diacritic restoration.