Continued Pretraining for Low-Resource Swahili ASR: Achieving State-of-the-Art Performance with Minimal Labeled Data
Not on the map. Why?
Abstract (arXiv)
We investigate continued pretraining (CPT) for adapting wav2vec2-bert-2.0 to Swahili automatic speech recognition (ASR). Our approach combines unlabeled audio with limited labeled data through pseudo-labeled CPT followed by supervised finetuning. With 20,000 labeled samples, we achieve 3.24% WER on Common Voice Swahili-an 82% relative improvement over the baseline. This result surpasses the best previously reported academic system (8.3% WER from XLS-R) by 61% relative improvement. We provide concrete data requirements and a replicable methodology applicable to other low-resource languages.
Key facts. Cited by 2 papers in the map (data through 7 Oct 2026). Builds on 9 of its 12 references that are in the map. First posted on arXiv 11 Mar 2026.
Builds on
9 of 12 references.
References · 9
- 2017Attention Is All You Needcited by 59,932
- 2020wav2vec 2.0: A Framework for Self-Supervised Learning of Speech Representationscited by 3,853
- 2019Common Voice: A Massively-Multilingual Speech Corpuscited by 1,358
- 2021XLS-R: Self-supervised Cross-lingual Speech Representation Learning at Scalecited by 589
- 2021SpeechBrain: A General-Purpose Speech Toolkitcited by 404
- 2023Seamless: Multilingual Expressive and Streaming Speech Translationcited by 190
- 2021Robust wav2vec 2.0: Analyzing Domain Shift in Self-Supervised Pre-Trainingcited by 148
- 2023Adapting Multilingual Speech Representation Model for a New, Underresourced Language through Multilingual Fine-tuning and Continued Pretrainingcited by 18
- 2024Continued Pretraining for Domain Adaptation of Wav2vec2.0 in Automatic Speech Recognition for Elementary Math Classroom Settingscited by 4
Built on by
Citing papers per year: 2026: 2.
2 papers.