# Findings of the 2023 ML-SUPERB Challenge: Pre-Training and Evaluation over More Languages and Beyond

arXiv: https://arxiv.org/abs/2310.05513 · HTML: https://mlsanity.com/paper/2310.05513/

## Key facts

- Cited by 10 papers in the map (data through 14 Oct 2026).
- Builds on 38 of its 61 references that are in the map.
- First posted on arXiv 9 Oct 2023.

## Builds on (most cited first, 30 of 38)

- [wav2vec 2.0: A Framework for Self-Supervised Learning of Speech Representations](https://mlsanity.com/paper/2006.11477/) (2020)
- [HuBERT: Self-Supervised Speech Representation Learning by Masked Prediction of Hidden Units](https://mlsanity.com/paper/2106.07447/) (2021)
- [WavLM: Large-Scale Self-Supervised Pre-Training for Full Stack Speech Processing](https://mlsanity.com/paper/2110.13900/) (2021)
- [Common Voice: A Massively-Multilingual Speech Corpus](https://mlsanity.com/paper/1912.06670/) (2019)
- [SUPERB: Speech processing Universal PERformance Benchmark](https://mlsanity.com/paper/2105.01051/) (2021)
- [XLS-R: Self-supervised Cross-lingual Speech Representation Learning at Scale](https://mlsanity.com/paper/2111.09296/) (2021)
- [Unsupervised Cross-lingual Representation Learning for Speech Recognition](https://mlsanity.com/paper/2006.13979/) (2020)
- [VoxPopuli: A Large-Scale Multilingual Speech Corpus for Representation Learning, Semi-Supervised Learning and Interpretation](https://mlsanity.com/paper/2101.00390/) (2021)
- [Scaling Speech Technology to 1,000+ Languages](https://mlsanity.com/paper/2305.13516/) (2023)
- [Self-Supervised Speech Representation Learning: A Review](https://mlsanity.com/paper/2205.10643/) (2022)
- [Textless Speech-to-Speech Translation on Real Data](https://mlsanity.com/paper/2112.08352/) (2021)
- [Massively Multilingual ASR: 50 Languages, 1 Model, 1 Billion Parameters](https://mlsanity.com/paper/2007.03001/) (2020)
- [ML-SUPERB: Multilingual Speech Universal PERformance Benchmark](https://mlsanity.com/paper/2305.10615/) (2023)
- [Scaling End-to-End Models for Large-Scale Multilingual ASR](https://mlsanity.com/paper/2104.14830/) (2021)
- [Multi-Singer: Fast Multi-Singer Singing Voice Vocoder With A Large-Scale Corpus](https://mlsanity.com/paper/2112.10358/) (2021)
- [ESPnet-SLU: Advancing Spoken Language Understanding through ESPnet](https://mlsanity.com/paper/2111.14706/) (2021)
- [Opencpop: A High-Quality Open Source Chinese Popular Song Corpus for Singing Voice Synthesis](https://mlsanity.com/paper/2201.07429/) (2022)
- [LeBenchmark: A Reproducible Framework for Assessing Self-Supervised Representation Learning from Speech](https://mlsanity.com/paper/2104.11462/) (2021)
- [IndicSUPERB: A Speech Processing Universal Performance Benchmark for Indian languages](https://mlsanity.com/paper/2208.11761/) (2022)
- [Learning Robust and Multilingual Speech Representations](https://mlsanity.com/paper/2001.11128/) (2020)
- [NusaCrowd: Open Source Initiative for Indonesian NLP Resources](https://mlsanity.com/paper/2212.09648/) (2022)
- [SpeechMatrix: A Large-Scale Mined Corpus of Multilingual Speech-to-Speech Translations](https://mlsanity.com/paper/2211.04508/) (2022)
- [Improving Massively Multilingual ASR With Auxiliary CTC Objectives](https://mlsanity.com/paper/2302.12829/) (2023)
- [ASR2K: Speech Recognition for Around 2000 Languages without Audio](https://mlsanity.com/paper/2209.02842/) (2022)
- [SUPERB @ SLT 2022: Challenge on Generalization and Efficiency of Self-Supervised Speech Representation Learning](https://mlsanity.com/paper/2210.08634/) (2022)
- [XTREME-S: Evaluating Cross-lingual Speech Representations](https://mlsanity.com/paper/2203.10752/) (2022)
- [Self-Supervised Representations Improve End-to-End Speech Translation](https://mlsanity.com/paper/2006.12124/) (2020)
- [Reducing Barriers to Self-Supervised Learning: HuBERT Pre-training with Academic Compute](https://mlsanity.com/paper/2306.06672/) (2023)
- [Improved Language Identification Through Cross-Lingual Self-Supervised Learning](https://mlsanity.com/paper/2107.04082/) (2021)
- [Leveraging End-to-End ASR for Endangered Language Documentation: An Empirical Study on Yolox'ochitl Mixtec](https://mlsanity.com/paper/2101.10877/) (2021)

## Built on by (most cited first, 10 of 10)

- [mHuBERT-147: A Compact Multilingual HuBERT Model](https://mlsanity.com/paper/2406.06371/) (2024)
- [Towards Robust Speech Representation Learning for Thousands of Languages](https://mlsanity.com/paper/2407.00837/) (2024)
- [ML-SUPERB 2.0: Benchmarking Multilingual Speech Models Across Modeling Constraints, Languages, and Datasets](https://mlsanity.com/paper/2406.08641/) (2024)
- [Wav2Gloss: Generating Interlinear Glossed Text from Speech](https://mlsanity.com/paper/2403.13169/) (2024)
- [The ML-SUPERB 2.0 Challenge: Towards Inclusive ASR Benchmarking for All Language Varieties](https://mlsanity.com/paper/2509.07139/) (2025)
- [CA-SSLR: Condition-Aware Self-Supervised Learning Representation for Generalized Speech Processing](https://mlsanity.com/paper/2412.04425/) (2024)
- [Speech-FT: Merging Pre-trained And Fine-Tuned Speech Representation Models For Cross-Task Generalization](https://mlsanity.com/paper/2502.12672/) (2025)
- [An Exploration of Mamba for Speech Self-Supervised Models](https://mlsanity.com/paper/2506.12606/) (2025)
- [The Faetar Benchmark: Speech Recognition in a Very Under-Resourced Language](https://mlsanity.com/paper/2409.08103/) (2024)
- [ESPnet-EZ: Python-only ESPnet for Easy Fine-tuning and Integration](https://mlsanity.com/paper/2409.09506/) (2024)

## Abstract (arXiv)

The 2023 Multilingual Speech Universal Performance Benchmark (ML-SUPERB) Challenge expands upon the acclaimed SUPERB framework, emphasizing self-supervised models in multilingual speech recognition and language identification. The challenge comprises a research track focused on applying ML-SUPERB to specific multilingual subjects, a Challenge Track for model submissions, and a New Language Track where language resource researchers can contribute and evaluate their low-resource language data in the context of the latest progress in multilingual speech recognition. The challenge garnered 12 model submissions and 54 language corpora, resulting in a comprehensive benchmark encompassing 154 languages. The findings indicate that merely scaling models is not the definitive solution for multilingual speech tasks, and a variety of speech/voice types present significant challenges in multilingual speech processing.