# Multilingual self-supervised speech representations improve the speech recognition of low-resource African languages with codeswitching

arXiv: https://arxiv.org/abs/2311.15077 · HTML: https://mlsanity.com/paper/2311.15077/

## Key facts

- Cited by 11 papers in the map (data through 14 Oct 2026).
- Builds on 6 of its 20 references that are in the map.
- First posted on arXiv 25 Nov 2023.

## Builds on (most cited first, 6 of 6)

- [wav2vec 2.0: A Framework for Self-Supervised Learning of Speech Representations](https://mlsanity.com/paper/2006.11477/) (2020)
- [Unsupervised Cross-lingual Representation Learning for Speech Recognition](https://mlsanity.com/paper/2006.13979/) (2020)
- [On the End-to-End Solution to Mandarin-English Code-switching Speech Recognition](https://mlsanity.com/paper/1811.00241/) (2018)
- [Towards End-to-End Code-Switching Speech Recognition](https://mlsanity.com/paper/1810.13091/) (2018)
- [Towards End-to-end Automatic Code-Switching Speech Recognition](https://mlsanity.com/paper/1810.12620/) (2018)
- [Code-Switching Detection Using ASR-Generated Language Posteriors](https://mlsanity.com/paper/1906.08003/) (2019)

## Built on by (most cited first, 11 of 11)

- [The NaijaVoices Dataset: Cultivating Large-Scale, High-Quality, Culturally-Rich Speech Data for African Languages](https://mlsanity.com/paper/2505.20564/) (2025)
- [SC-MoE: Switch Conformer Mixture of Experts for Unified Streaming and Non-streaming Code-Switching ASR](https://mlsanity.com/paper/2406.18021/) (2024)
- [Improving Cross-Lingual Phonetic Representation of Low-Resource Languages Through Language Similarity Analysis](https://mlsanity.com/paper/2501.06810/) (2025)
- [Code-Switching in End-to-End Automatic Speech Recognition: A Systematic Literature Review](https://mlsanity.com/paper/2507.07741/) (2025)
- [CAMEL: Cross-Attention Enhanced Mixture-of-Experts and Language Bias for Code-Switching Speech Recognition](https://mlsanity.com/paper/2412.12760/) (2024)
- [SAGE: Spliced-Audio Generated Data for Enhancing Foundational Models in Low-Resource Arabic-English Code-Switched Speech Recognition](https://mlsanity.com/paper/2506.22143/) (2025)
- [How much speech data is necessary for ASR in African languages? An evaluation of data scaling in Kinyarwanda and Kikuyu](https://mlsanity.com/paper/2510.07221/) (2025)
- [AfroScope: A Framework for Studying the Linguistic Landscape of Africa](https://mlsanity.com/paper/2601.13346/) (2026)
- [Neither Here Nor There: Cross-Lingual Representation Dynamics of Code-Mixed Text in Multilingual Encoders](https://mlsanity.com/paper/2603.19771/) (2026)
- [Tone-Conditioned Curriculum Learning for Low-Resource Bantu Speech Recognition](https://mlsanity.com/paper/2606.31642/) (2026)
- [AfriSwitch: A Benchmark for In-the-Wild African Code-Switched Speech Recognition](https://mlsanity.com/paper/2608.26434/) (2026)

## Abstract (arXiv)

While many speakers of low-resource languages regularly code-switch between their languages and other regional languages or English, datasets of codeswitched speech are too small to train bespoke acoustic models from scratch or do language model rescoring. Here we propose finetuning self-supervised speech representations such as wav2vec 2.0 XLSR to recognize code-switched data. We find that finetuning self-supervised multilingual representations and augmenting them with n-gram language models trained from transcripts reduces absolute word error rates by up to 20% compared to baselines of hybrid models trained from scratch on code-switched data. Our findings suggest that in circumstances with limited training data finetuning self-supervised representations is a better performing and viable solution.