# Phoneme Recognition through Fine Tuning of Phonetic Representations: a Case Study on Luhya Language Varieties

arXiv: https://arxiv.org/abs/2104.01624 · HTML: https://mlsanity.com/paper/2104.01624/

## Key facts

- Cited by 2 papers in the map (data through 14 Oct 2026).
- Builds on 7 of its 18 references that are in the map.
- First posted on arXiv 4 Apr 2021.

## Builds on (most cited first, 7 of 7)

- [wav2vec 2.0: A Framework for Self-Supervised Learning of Speech Representations](https://mlsanity.com/paper/2006.11477/) (2020)
- [Rapid Adaptation of Neural Machine Translation to New Languages](https://mlsanity.com/paper/1808.04189/) (2018)
- [Universal Phone Recognition with a Multilingual Allophone System](https://mlsanity.com/paper/2002.11800/) (2020)
- [Pre-training on high-resource speech recognition improves low-resource speech-to-text translation](https://mlsanity.com/paper/1809.01431/) (2018)
- [Meta Learning for End-to-End Low-Resource Speech Recognition](https://mlsanity.com/paper/1910.12094/) (2019)
- [Analyzing ASR pretraining for low-resource speech-to-text translation](https://mlsanity.com/paper/1910.10762/) (2019)
- [Improving Cross-Lingual Transfer Learning for End-to-End Speech Recognition with Speech Translation](https://mlsanity.com/paper/2006.05474/) (2020)

## Built on by (most cited first, 2 of 2)

- [Voices Unheard: NLP Resources and Models for Yor`ub'a Regional Dialects](https://mlsanity.com/paper/2406.19564/) (2024)
- [Hard to Be Heard: Phoneme-Level ASR Analysis of Phonologically Complex, Low-Resource Endangered Languages](https://mlsanity.com/paper/2604.18204/) (2026)

## Abstract (arXiv)

Models pre-trained on multiple languages have shown significant promise for improving speech recognition, particularly for low-resource languages. In this work, we focus on phoneme recognition using Allosaurus, a method for multilingual recognition based on phonetic annotation, which incorporates phonological knowledge through a language-dependent allophone layer that associates a universal narrow phone-set with the phonemes that appear in each language. To evaluate in a challenging real-world scenario, we curate phone recognition datasets for Bukusu and Saamia, two varieties of the Luhya language cluster of western Kenya and eastern Uganda. To our knowledge, these datasets are the first of their kind. We carry out similar experiments on the dataset of an endangered Tangkhulic language, East Tusom, a Tibeto-Burman language variety spoken mostly in India. We explore both zero-shot and few-shot recognition by fine-tuning using datasets of varying sizes (10 to 1000 utterances). We find that fine-tuning of Allosaurus, even with just 100 utterances, leads to significant improvements in phone error rates.