# Massively Multilingual ASR: 50 Languages, 1 Model, 1 Billion Parameters

arXiv: https://arxiv.org/abs/2007.03001 · HTML: https://mlsanity.com/paper/2007.03001/

## Key facts

- Cited by 110 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 6 Jul 2020.

## Built on by (most cited first, 30 of 110)

- [Robust Speech Recognition via Large-Scale Weak Supervision](https://mlsanity.com/paper/2212.04356/) (2022)
- [Finetuned Language Models Are Zero-Shot Learners](https://mlsanity.com/paper/2109.01652/) (2021)
- [A Generalist Agent](https://mlsanity.com/paper/2205.06175/) (2022)
- [Scaling Speech Technology to 1,000+ Languages](https://mlsanity.com/paper/2305.13516/) (2023)
- [Recent Advances in End-to-End Automatic Speech Recognition](https://mlsanity.com/paper/2111.01690/) (2021)
- [End-to-End Speech Recognition: A Survey](https://mlsanity.com/paper/2303.03329/) (2023)
- [Simple and Effective Zero-shot Cross-lingual Phoneme Recognition](https://mlsanity.com/paper/2109.11680/) (2021)
- [Scaling End-to-End Models for Large-Scale Multilingual ASR](https://mlsanity.com/paper/2104.14830/) (2021)
- [Multilingual and code-switching ASR challenges for low resource Indian languages](https://mlsanity.com/paper/2104.00235/) (2021)
- [PARP: Prune, Adjust and Re-Prune for Self-Supervised Speech Recognition](https://mlsanity.com/paper/2106.05933/) (2021)
- [Towards One Model to Rule All: Multilingual Strategy for Dialectal Code-Switching Arabic ASR](https://mlsanity.com/paper/2105.14779/) (2021)
- [u-HuBERT: Unified Mixed-Modal Speech Pretraining And Zero-Shot Transfer to Unlabeled Modality](https://mlsanity.com/paper/2207.07036/) (2022)
- [Multilingual and Cross-Lingual Intent Detection from Spoken Data](https://mlsanity.com/paper/2104.08524/) (2021)
- [Exploiting Adapters for Cross-lingual Low-resource Speech Recognition](https://mlsanity.com/paper/2105.11905/) (2021)
- [GigaSpeech 2: An Evolving, Large-Scale and Multi-domain ASR Corpus for Low-Resource Languages with Automated Crawling, Transcription and Refinement](https://mlsanity.com/paper/2406.11546/) (2024)
- [Improving Massively Multilingual ASR With Auxiliary CTC Objectives](https://mlsanity.com/paper/2302.12829/) (2023)
- [Large Language Model Instruction Following: A Survey of Progresses and Challenges](https://mlsanity.com/paper/2303.10475/) (2023)
- [A Configurable Multilingual Model is All You Need to Recognize All Languages](https://mlsanity.com/paper/2107.05876/) (2021)
- [ML-SUPERB 2.0: Benchmarking Multilingual Speech Models Across Modeling Constraints, Languages, and Datasets](https://mlsanity.com/paper/2406.08641/) (2024)
- [Pseudo-Labeling for Massively Multilingual Speech Recognition](https://mlsanity.com/paper/2111.00161/) (2021)
- [Streaming End-to-End Multilingual Speech Recognition with Joint Language Identification](https://mlsanity.com/paper/2209.06058/) (2022)
- [LAE: Language-Aware Encoder for Monolingual and Multilingual ASR](https://mlsanity.com/paper/2206.02093/) (2022)
- [A Survey of Multilingual Models for Automatic Speech Recognition](https://mlsanity.com/paper/2202.12576/) (2022)
- [Improved Language Identification Through Cross-Lingual Self-Supervised Learning](https://mlsanity.com/paper/2107.04082/) (2021)
- [Comparison of Multilingual Self-Supervised and Weakly-Supervised Speech Pre-Training for Adaptation to Unseen Languages](https://mlsanity.com/paper/2305.12606/) (2023)
- [Joint Modeling of Code-Switched and Monolingual ASR via Conditional Factorization](https://mlsanity.com/paper/2111.15016/) (2021)
- [Crossing the Conversational Chasm: A Primer on Natural Language Processing for Multilingual Task-Oriented Dialogue Systems](https://mlsanity.com/paper/2104.08570/) (2021)
- [Mixture-of-Expert Conformer for Streaming Multilingual ASR](https://mlsanity.com/paper/2305.15663/) (2023)
- [Whistle: Data-Efficient Multilingual and Crosslingual Speech Recognition via Weakly Phonetic Supervision](https://mlsanity.com/paper/2406.02166/) (2024)
- [Massively Multilingual ASR on 70 Languages: Tokenization, Architecture, and Generalization Capabilities](https://mlsanity.com/paper/2211.05756/) (2022)

## Abstract (arXiv)

We study training a single acoustic model for multiple languages with the aim of improving automatic speech recognition (ASR) performance on low-resource languages, and over-all simplifying deployment of ASR systems that support diverse languages. We perform an extensive benchmark on 51 languages, with varying amount of training data by language(from 100 hours to 1100 hours). We compare three variants of multilingual training from a single joint model without knowing the input language, to using this information, to multiple heads (one per language cluster). We show that multilingual training of ASR models on several languages can improve recognition performance, in particular, on low resource languages. We see 20.9%, 23% and 28.8% average WER relative reduction compared to monolingual baselines on joint model, joint model with language input and multi head model respectively. To our knowledge, this is the first work studying multilingual ASR at massive scale, with more than 50 languages and more than 16,000 hours of audio across them.