# BuzzASR: A Swarm of 100+ Monolingual Speech Recognition Models

arXiv: https://arxiv.org/abs/2609.09554 · HTML: https://mlsanity.com/paper/2609.09554/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 30 of its 57 references that are in the map.
- First posted on arXiv 9 Sep 2026.

## Builds on (most cited first, 30 of 30)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [Robust Speech Recognition via Large-Scale Weak Supervision](https://mlsanity.com/paper/2212.04356/) (2022)
- [Unsupervised Cross-lingual Representation Learning at Scale](https://mlsanity.com/paper/1911.02116/) (2019)
- [wav2vec 2.0: A Framework for Self-Supervised Learning of Speech Representations](https://mlsanity.com/paper/2006.11477/) (2020)
- [Neural Machine Translation of Rare Words with Subword Units](https://mlsanity.com/paper/1508.07909/) (2015)
- [Common Voice: A Massively-Multilingual Speech Corpus](https://mlsanity.com/paper/1912.06670/) (2019)
- [XLS-R: Self-supervised Cross-lingual Speech Representation Learning at Scale](https://mlsanity.com/paper/2111.09296/) (2021)
- [The FLORES-101 Evaluation Benchmark for Low-Resource and Multilingual Machine Translation](https://mlsanity.com/paper/2106.03193/) (2021)
- [MLS: A Large-Scale Multilingual Dataset for Speech Research](https://mlsanity.com/paper/2012.03411/) (2020)
- [FLEURS: Few-shot Learning Evaluation of Universal Representations of Speech](https://mlsanity.com/paper/2205.12446/) (2022)
- [VoxPopuli: A Large-Scale Multilingual Speech Corpus for Representation Learning, Semi-Supervised Learning and Interpretation](https://mlsanity.com/paper/2101.00390/) (2021)
- [Scaling Speech Technology to 1,000+ Languages](https://mlsanity.com/paper/2305.13516/) (2023)
- [WECHSEL: Effective initialization of subword embeddings for cross-lingual transfer of monolingual language models](https://mlsanity.com/paper/2112.06598/) (2021)
- [Multilingual Speech Recognition With A Single End-To-End Model](https://mlsanity.com/paper/1711.01694/) (2017)
- [Multilingual and code-switching ASR challenges for low resource Indian languages](https://mlsanity.com/paper/2104.00235/) (2021)
- [Towards Robust Speech Representation Learning for Thousands of Languages](https://mlsanity.com/paper/2407.00837/) (2024)
- [Fast Vocabulary Transfer for Language Model Compression](https://mlsanity.com/paper/2402.09977/) (2024)
- [AfriSpeech-200: Pan-African Accented Speech Dataset for Clinical and General Domain ASR](https://mlsanity.com/paper/2310.00274/) (2023)
- [Goldfish: Monolingual Language Models for 350 Languages](https://mlsanity.com/paper/2408.10441/) (2024)
- [Vistaar: Diverse Benchmarks and Training Sets for Indian Language ASR](https://mlsanity.com/paper/2305.15386/) (2023)
- [N-Shot Benchmarking of Whisper on Diverse Arabic Speech Recognition](https://mlsanity.com/paper/2306.02902/) (2023)
- [The National Research Platform: Stretched, Multi-Tenant, Scientific Kubernetes Cluster](https://mlsanity.com/paper/2505.22864/) (2025)
- [Extending Whisper with prompt tuning to target-speaker ASR](https://mlsanity.com/paper/2312.08079/) (2023)
- [MSR-86K: An Evolving, Multilingual Corpus with 86,300 Hours of Transcribed Audio for Speech Recognition Research](https://mlsanity.com/paper/2406.18301/) (2024)
- [Overcoming Data Scarcity in Multi-Dialectal Arabic ASR via Whisper Fine-Tuning](https://mlsanity.com/paper/2506.02627/) (2025)
- [Low-Rank and Sparse Model Merging for Multi-Lingual Speech Recognition and Translation](https://mlsanity.com/paper/2502.17380/) (2025)
- [Automatic Speech Recognition for African Low-Resource Languages: Challenges and Future Directions](https://mlsanity.com/paper/2505.11690/) (2025)
- [Adapting Whisper for Parameter-efficient Code-Switching Speech Recognition via Soft Prompt Tuning](https://mlsanity.com/paper/2506.21576/) (2025)
- [What Do Prosody and Text Convey? Characterizing How Meaningful Information is Distributed Across Multiple Channels](https://mlsanity.com/paper/2512.16832/) (2025)
- [Zipper-LoRA: Dynamic Parameter Decoupling for Speech-LLM based Multilingual Speech Recognition](https://mlsanity.com/paper/2603.17558/) (2026)

## Abstract (arXiv)

We introduce BuzzASR, a collection of language-specialized fine-tuned Whisper models adapted for automatic speech recognition (ASR) in 102 languages. Large end-to-end Transformer-based ASR models such as Whisper have revolutionized ASR, but most prominent models are highly multilingual. As a result, these models often perform poorly on languages less well-represented in their training set. While it has long been known that effective language adaptation can be achieved through simple fine-tuning on monolingual data, this strategy has only been applied to a small number of languages. We massively scale up this simple approach to 102 languages covered in the FLEURS dataset, while also implementing a more complex language adaptation strategy that integrates monolingual tokenizer replacement and data augmentation using text-only fine-tuning. BuzzASR models outperform Whisper-large-v3 on 77 out of 102 languages, reducing character error rates (CER) by a factor of over 2.8 on average. Our models achieve state-of-the-art CER among open-source systems on 27 of 102 languages on the combined FLEURS and Common Voice test set. Our tokenizer replacement strategy yields an average 3.3x improvement in compression rate (characters per token) over Whisper's multilingual BPE, with gains of up to 21.7x. We release all models, code, and detailed results: https://lemn-lab.github.io/buzz-asr