# Fast Development of ASR in African Languages using Self Supervised Speech Representation Learning

arXiv: https://arxiv.org/abs/2103.08993 · HTML: https://mlsanity.com/paper/2103.08993/

## Key facts

- Cited by 3 papers in the map (data through 14 Oct 2026).
- Builds on 19 of its 28 references that are in the map.
- First posted on arXiv 16 Mar 2021.

## Builds on (most cited first, 19 of 19)

- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [A Simple Framework for Contrastive Learning of Visual Representations](https://mlsanity.com/paper/2002.05709/) (2020)
- [Representation Learning with Contrastive Predictive Coding](https://mlsanity.com/paper/1807.03748/) (2018)
- [Momentum Contrast for Unsupervised Visual Representation Learning](https://mlsanity.com/paper/1911.05722/) (2019)
- [wav2vec 2.0: A Framework for Self-Supervised Learning of Speech Representations](https://mlsanity.com/paper/2006.11477/) (2020)
- [Deep contextualized word representations](https://mlsanity.com/paper/1802.05365/) (2018)
- [Self-Supervised Learning of Pretext-Invariant Representations](https://mlsanity.com/paper/1912.01991/) (2019)
- [Learning Representations by Maximizing Mutual Information Across Views](https://mlsanity.com/paper/1906.00910/) (2019)
- [Data-Efficient Image Recognition with Contrastive Predictive Coding](https://mlsanity.com/paper/1905.09272/) (2019)
- [Libri-Light: A Benchmark for ASR with Limited or No Supervision](https://mlsanity.com/paper/1912.07875/) (2019)
- [An Unsupervised Autoregressive Model for Speech Representation Learning](https://mlsanity.com/paper/1904.03240/) (2019)
- [DDSP: Differentiable Digital Signal Processing](https://mlsanity.com/paper/2001.04643/) (2020)
- [Multi-task self-supervised learning for Robust Speech Recognition](https://mlsanity.com/paper/2001.09239/) (2020)
- [Unsupervised pretraining transfers well across languages](https://mlsanity.com/paper/2002.02848/) (2020)
- [Learning Problem-agnostic Speech Representations from Multiple Self-supervised Tasks](https://mlsanity.com/paper/1904.03416/) (2019)
- [Generative Pre-Training for Speech with Autoregressive Predictive Coding](https://mlsanity.com/paper/1910.12607/) (2019)
- [Effectiveness of self-supervised pre-training for speech recognition](https://mlsanity.com/paper/1911.03912/) (2019)
- [Learning Robust and Multilingual Speech Representations](https://mlsanity.com/paper/2001.11128/) (2020)
- [Improved Speech Representations with Multi-Target Autoregressive Predictive Coding](https://mlsanity.com/paper/2004.05274/) (2020)

## Built on by (most cited first, 3 of 3)

- [Charting the Landscape of African NLP: Mapping Progress and Shaping the Road Ahead](https://mlsanity.com/paper/2505.21315/) (2025)
- [Preuve de concept d'un bot vocal dialoguant en wolof](https://mlsanity.com/paper/2404.02009/) (2024)
- [Automatic Speech Recognition (ASR) for African Low-Resource Languages: A Systematic Literature Review](https://mlsanity.com/paper/2510.01145/) (2025)

## Abstract (arXiv)

This paper describes the results of an informal collaboration launched during the African Master of Machine Intelligence (AMMI) in June 2020. After a series of lectures and labs on speech data collection using mobile applications and on self-supervised representation learning from speech, a small group of students and the lecturer continued working on automatic speech recognition (ASR) project for three languages: Wolof, Ga, and Somali. This paper describes how data was collected and ASR systems developed with a small amount (1h) of transcribed speech as training data. In these low resource conditions, pre-training a model on large amounts of raw speech was fundamental for the efficiency of ASR systems developed.