# MUST: A Multilingual Student-Teacher Learning approach for low-resource speech recognition

arXiv: https://arxiv.org/abs/2310.18865 · HTML: https://mlsanity.com/paper/2310.18865/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Placed in Speech and audio distillation, one of 12 directions in Knowledge distillation.
- Builds on 16 of its 30 references that are in the map.
- First posted on arXiv 29 Oct 2023.

## Placed in

- Knowledge distillation / [Speech and audio distillation](https://mlsanity.com/distill/speech-and-audio-distillation/)

## Builds on (most cited first, 16 of 16)

- [Distilling the Knowledge in a Neural Network](https://mlsanity.com/paper/1503.02531/) (2015)
- [FitNets: Hints for Thin Deep Nets](https://mlsanity.com/paper/1412.6550/) (2014)
- [Subword Regularization: Improving Neural Network Translation Models with Multiple Subword Candidates](https://mlsanity.com/paper/1804.10959/) (2018)
- [SpeechBrain: A General-Purpose Speech Toolkit](https://mlsanity.com/paper/2106.04624/) (2021)
- [Joint CTC-Attention based End-to-End Speech Recognition using Multi-task Learning](https://mlsanity.com/paper/1609.06773/) (2016)
- [Massively Multilingual ASR: 50 Languages, 1 Model, 1 Billion Parameters](https://mlsanity.com/paper/2007.03001/) (2020)
- [Simple and Effective Zero-shot Cross-lingual Phoneme Recognition](https://mlsanity.com/paper/2109.11680/) (2021)
- [Mosaicking to Distill: Knowledge Distillation from Out-of-Domain Data](https://mlsanity.com/paper/2110.15094/) (2021)
- [Embracing the Dark Knowledge: Domain Generalization Using Regularized Knowledge Distillation](https://mlsanity.com/paper/2107.02629/) (2021)
- [Ensemble knowledge distillation of self-supervised speech models](https://mlsanity.com/paper/2302.12757/) (2023)
- [Deciphering Speech: a Zero-Resource Approach to Cross-Lingual Transfer in ASR](https://mlsanity.com/paper/2111.06799/) (2021)
- [Domain Generalization on Efficient Acoustic Scene Classification using Residual Normalization](https://mlsanity.com/paper/2111.06531/) (2021)
- [Investigating the Impact of Cross-lingual Acoustic-Phonetic Similarities on Multilingual Speech Recognition](https://mlsanity.com/paper/2207.03390/) (2022)
- [Non-Linear Pairwise Language Mappings for Low-Resource Multilingual Acoustic Model Fusion](https://mlsanity.com/paper/2207.03391/) (2022)
- [Towards domain generalisation in ASR with elitist sampling and ensemble knowledge distillation](https://mlsanity.com/paper/2303.00550/) (2023)
- [Learning Cross-lingual Mappings for Data Augmentation to Improve Low-Resource Speech Recognition](https://mlsanity.com/paper/2306.08577/) (2023)

## Abstract (arXiv)

Student-teacher learning or knowledge distillation (KD) has been previously used to address data scarcity issue for training of speech recognition (ASR) systems. However, a limitation of KD training is that the student model classes must be a proper or improper subset of the teacher model classes. It prevents distillation from even acoustically similar languages if the character sets are not same. In this work, the aforementioned limitation is addressed by proposing a MUltilingual Student-Teacher (MUST) learning which exploits a posteriors mapping approach. A pre-trained mapping model is used to map posteriors from a teacher language to the student language ASR. These mapped posteriors are used as soft labels for KD learning. Various teacher ensemble schemes are experimented to train an ASR model for low-resource languages. A model trained with MUST learning reduces relative character error rate (CER) up to 9.5% in comparison with a baseline monolingual ASR.