Fast Development of ASR in African Languages using Self Supervised Speech Representation Learning
Not on the map. Why?
Abstract (arXiv)
This paper describes the results of an informal collaboration launched during the African Master of Machine Intelligence (AMMI) in June 2020. After a series of lectures and labs on speech data collection using mobile applications and on self-supervised representation learning from speech, a small group of students and the lecturer continued working on automatic speech recognition (ASR) project for three languages: Wolof, Ga, and Somali. This paper describes how data was collected and ASR systems developed with a small amount (1h) of transcribed speech as training data. In these low resource conditions, pre-training a model on large amounts of raw speech was fundamental for the efficiency of ASR systems developed.
Key facts. Cited by 3 papers in the map (data through 7 Oct 2026). Builds on 19 of its 28 references that are in the map. First posted on arXiv 16 Mar 2021.
Builds on
19 of 28 references.
References · 19
- 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,204
- 2020A Simple Framework for Contrastive Learning of Visual Representationscited by 11,979
- 2018Representation Learning with Contrastive Predictive Codingcited by 7,662
- 2019Momentum Contrast for Unsupervised Visual Representation Learningcited by 7,482
- 2020wav2vec 2.0: A Framework for Self-Supervised Learning of Speech Representationscited by 3,853
- 2018Deep contextualized word representationscited by 2,285
- 2019Self-Supervised Learning of Pretext-Invariant Representationscited by 806
- 2019Learning Representations by Maximizing Mutual Information Across Viewscited by 733
- 2019Data-Efficient Image Recognition with Contrastive Predictive Codingcited by 672
- 2019Libri-Light: A Benchmark for ASR with Limited or No Supervisioncited by 462
- 2019An Unsupervised Autoregressive Model for Speech Representation Learningcited by 185
- 2020DDSP: Differentiable Digital Signal Processingcited by 177
7 more
- 2020Multi-task self-supervised learning for Robust Speech Recognitioncited by 127
- 2020Unsupervised pretraining transfers well across languagescited by 100
- 2019Learning Problem-agnostic Speech Representations from Multiple Self-supervised Taskscited by 87
- 2019Generative Pre-Training for Speech with Autoregressive Predictive Codingcited by 82
- 2019Effectiveness of self-supervised pre-training for speech recognitioncited by 72
- 2020Learning Robust and Multilingual Speech Representationscited by 39
- 2020Improved Speech Representations with Multi-Target Autoregressive Predictive Codingcited by 23
Built on by
Citing papers per year: 2024: 1; 2025: 2.
3 papers.