# The Challenge of Diacritics in Yoruba Embeddings

arXiv: https://arxiv.org/abs/2011.07605 · HTML: https://mlsanity.com/paper/2011.07605/

## Key facts

- Cited by 3 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 15 Nov 2020.

## Built on by (most cited first, 3 of 3)

- [State-of-the-art in Open-domain Conversational AI: A Survey](https://mlsanity.com/paper/2205.00965/) (2022)
- [AfriWOZ: Corpus for Exploiting Cross-Lingual Transferability for Generation of Dialogues in Low-Resource, African Languages](https://mlsanity.com/paper/2204.08083/) (2022)
- [NaijaNLP: A Survey of Nigerian Low-Resource Languages](https://mlsanity.com/paper/2502.19784/) (2025)

## Abstract (arXiv)

The major contributions of this work include the empirical establishment of a better performance for Yoruba embeddings from undiacritized (normalized) dataset and provision of new analogy sets for evaluation. The Yoruba language, being a tonal language, utilizes diacritics (tonal marks) in written form. We show that this affects embedding performance by creating embeddings from exactly the same Wikipedia dataset but with the second one normalized to be undiacritized. We further compare average intrinsic performance with two other work (using analogy test set & WordSim) and we obtain the best performance in WordSim and corresponding Spearman correlation.