# Using heterogeneity in semi-supervised transcription hypotheses to improve code-switched speech recognition

arXiv: https://arxiv.org/abs/2106.07699 · HTML: https://mlsanity.com/paper/2106.07699/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 4 of its 20 references that are in the map.
- First posted on arXiv 14 Jun 2021.

## Builds on (most cited first, 4 of 4)

- [A Survey of Code-switched Speech and Language Processing](https://mlsanity.com/paper/1904.00784/) (2019)
- [Semi-supervised acoustic model training for five-lingual code-switched ASR](https://mlsanity.com/paper/1906.08647/) (2019)
- [Study of Semi-supervised Approaches to Improving English-Mandarin Code-Switching Speech Recognition](https://mlsanity.com/paper/1806.06200/) (2018)
- [Semi-supervised acoustic model training for speech with code-switching](https://mlsanity.com/paper/1810.09699/) (2018)

## Abstract (arXiv)

Modeling code-switched speech is an important problem in automatic speech recognition (ASR). Labeled code-switched data are rare, so monolingual data are often used to model code-switched speech. These monolingual data may be more closely matched to one of the languages in the code-switch pair. We show that such asymmetry can bias prediction toward the better-matched language and degrade overall model performance. To address this issue, we propose a semi-supervised approach for code-switched ASR. We consider the case of English-Mandarin code-switching, and the problem of using monolingual data to build bilingual "transcription models'' for annotation of unlabeled code-switched data. We first build multiple transcription models so that their individual predictions are variously biased toward either English or Mandarin. We then combine these biased transcriptions using confidence-based selection. This strategy generates a superior transcript for semi-supervised training, and obtains a 19% relative improvement compared to a semi-supervised system that relies on a transcription model built with only the best-matched monolingual data.