# Text Normalization for Low-Resource Languages of Africa

arXiv: https://arxiv.org/abs/2103.15845 · HTML: https://mlsanity.com/paper/2103.15845/

## Key facts

- Cited by 2 papers in the map (data through 14 Oct 2026).
- Builds on 0 of its 16 references that are in the map.
- First posted on arXiv 29 Mar 2021.

## Built on by (most cited first, 2 of 2)

- [Large vocabulary speech recognition for languages of Africa: multilingual modeling and self-supervised learning](https://mlsanity.com/paper/2208.03067/) (2022)
- [Text normalization for low-resource languages: the case of Ligurian](https://mlsanity.com/paper/2206.07861/) (2022)

## Abstract (arXiv)

Training data for machine learning models can come from many different sources, which can be of dubious quality. For resource-rich languages like English, there is a lot of data available, so we can afford to throw out the dubious data. For low-resource languages where there is much less data available, we can't necessarily afford to throw out the dubious data, in case we end up with a training set which is too small to train a model. In this study, we examine the effects of text normalization and data set quality for a set of low-resource languages of Africa -- Afrikaans, Amharic, Hausa, Igbo, Malagasy, Somali, Swahili, and Zulu. We describe our text normalizer which we built in the Pynini framework, a Python library for finite state transducers, and our experiments in training language models for African languages using the Natural Language Toolkit (NLTK), an open-source Python library for NLP.