# Massive vs. Curated Word Embeddings for Low-Resourced Languages. The Case of Yor`ub'a and Twi

arXiv: https://arxiv.org/abs/1912.02481 · HTML: https://mlsanity.com/paper/1912.02481/

## Key facts

- Cited by 33 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 5 Dec 2019.

## Built on by (most cited first, 30 of 33)

- [MasakhaNER: Named Entity Recognition for African Languages](https://mlsanity.com/paper/2103.11811/) (2021)
- [Adapting Pre-trained Language Models to African Languages via Multilingual Adaptive Fine-Tuning](https://mlsanity.com/paper/2204.06487/) (2022)
- [NaijaSenti: A Nigerian Twitter Sentiment Corpus for Multilingual Sentiment Analysis](https://mlsanity.com/paper/2201.08277/) (2022)
- [SERENGETI: Massively Multilingual Language Models for Africa](https://mlsanity.com/paper/2212.10785/) (2022)
- [AfroLID: A Neural Language Identification Tool for African Languages](https://mlsanity.com/paper/2210.11744/) (2022)
- [The Effect of Domain and Diacritics in Yor`ub'a-English Neural Machine Translation](https://mlsanity.com/paper/2103.08647/) (2021)
- [MasakhaNER 2.0: Africa-centric Transfer Learning for Named Entity Recognition](https://mlsanity.com/paper/2210.12391/) (2022)
- [NBIAS: A Natural Language Processing Framework for Bias Identification in Text](https://mlsanity.com/paper/2308.01681/) (2023)
- [Charting the Landscape of African NLP: Mapping Progress and Shaping the Road Ahead](https://mlsanity.com/paper/2505.21315/) (2025)
- [yosm: A new yoruba sentiment corpus for movie reviews](https://mlsanity.com/paper/2204.09711/) (2022)
- [The Zeno's Paradox of `Low-Resource' Languages](https://mlsanity.com/paper/2410.20817/) (2024)
- [MMTAfrica: Multilingual Machine Translation for African Languages](https://mlsanity.com/paper/2204.04306/) (2022)
- [TaTa: A Multilingual Table-to-Text Dataset for African Languages](https://mlsanity.com/paper/2211.00142/) (2022)
- [Where Are We? Evaluating LLM Performance on African Languages](https://mlsanity.com/paper/2502.19582/) (2025)
- [ANEA: Distant Supervision for Low-Resource Named Entity Recognition](https://mlsanity.com/paper/2102.13129/) (2021)
- [Toucan: Many-to-Many Translation for 150 African Language Pairs](https://mlsanity.com/paper/2407.04796/) (2024)
- [English-Twi Parallel Corpus for Machine Translation](https://mlsanity.com/paper/2103.15625/) (2021)
- [Separating Grains from the Chaff: Using Data Filtering to Improve Multilingual Translation for Low-Resourced African Languages](https://mlsanity.com/paper/2210.10692/) (2022)
- [Akan Cinematic Emotions (ACE): A Multimodal Multi-party Dataset for Emotion Recognition in Movie Dialogues](https://mlsanity.com/paper/2502.10973/) (2025)
- [Training Cross-Lingual embeddings for Setswana and Sepedi](https://mlsanity.com/paper/2111.06230/) (2021)
- [Comparing LLM prompting with Cross-lingual transfer performance on Indigenous and Low-resource Brazilian Languages](https://mlsanity.com/paper/2404.18286/) (2024)
- [Fine-tuning Large Language Models with Limited Data: A Survey and Practical Guide](https://mlsanity.com/paper/2411.09539/) (2024)
- [AfriVEC: Word Embedding Models for African Languages. Case Study of Fon and Nobiin](https://mlsanity.com/paper/2103.05132/) (2021)
- [How Good is Your Wikipedia? Auditing Data Quality for Low-resource and Multilingual NLP](https://mlsanity.com/paper/2411.05527/) (2024)
- [Bridging Gaps in Natural Language Processing for Yor`ub'a: A Systematic Review of a Decade of Progress and Prospects](https://mlsanity.com/paper/2502.17364/) (2025)
- [Contextual Text Embeddings for Twi](https://mlsanity.com/paper/2103.15963/) (2021)
- [Few-Shot Cross-Lingual Transfer for Prompting Large Language Models in Low-Resource Languages](https://mlsanity.com/paper/2403.06018/) (2024)
- [The Impact of Data Corruption on Named Entity Recognition for Low-resourced Languages](https://mlsanity.com/paper/2208.04568/) (2022)
- [Is Translation Helpful? An Empirical Analysis of Cross-Lingual Transfer in Low-Resource Dialog Generation](https://mlsanity.com/paper/2305.12480/) (2023)
- [Yankari: A Monolingual Yoruba Dataset](https://mlsanity.com/paper/2412.03334/) (2024)

## Abstract (arXiv)

The success of several architectures to learn semantic representations from unannotated text and the availability of these kind of texts in online multilingual resources such as Wikipedia has facilitated the massive and automatic creation of resources for multiple languages. The evaluation of such resources is usually done for the high-resourced languages, where one has a smorgasbord of tasks and test sets to evaluate on. For low-resourced languages, the evaluation is more difficult and normally ignored, with the hope that the impressive capability of deep learning architectures to learn (multilingual) representations in the high-resourced setting holds in the low-resourced setting too. In this paper we focus on two African languages, Yor`ub'a and Twi, and compare the word embeddings obtained in this way, with word embeddings obtained from curated corpora and a language-dependent processing. We analyse the noise in the publicly available corpora, collect high quality and noisy data for the two languages and quantify the improvements that depend not only on the amount of data but on the quality too. We also use different architectures that learn word representations both from surface forms and characters to further exploit all the available information which showed to be important for these languages. For the evaluation, we manually translate the wordsim-353 word pairs dataset from English into Yor`ub'a and Twi. As output of the work, we provide corpora, embeddings and the test suits for both languages.