mlsanityData to 7 Oct 2026

Findings of the 2023 ML-SUPERB Challenge: Pre-Training and Evaluation over More Languages and Beyond

arXiv 2310.05513 ↗First posted 9 Oct 2023cs.SD cs.CL eess.AScited by 10

Not on the map. Why?

Abstract (arXiv)

The 2023 Multilingual Speech Universal Performance Benchmark (ML-SUPERB) Challenge expands upon the acclaimed SUPERB framework, emphasizing self-supervised models in multilingual speech recognition and language identification. The challenge comprises a research track focused on applying ML-SUPERB to specific multilingual subjects, a Challenge Track for model submissions, and a New Language Track where language resource researchers can contribute and evaluate their low-resource language data in the context of the latest progress in multilingual speech recognition. The challenge garnered 12 model submissions and 54 language corpora, resulting in a comprehensive benchmark encompassing 154 languages. The findings indicate that merely scaling models is not the definitive solution for multilingual speech tasks, and a variety of speech/voice types present significant challenges in multilingual speech processing.

Key facts. Cited by 10 papers in the map (data through 7 Oct 2026). Builds on 38 of its 61 references that are in the map. First posted on arXiv 9 Oct 2023.

Builds on

38 of 61 references.

References · 38

  1. 2020wav2vec 2.0: A Framework for Self-Supervised Learning of Speech Representationscited by 3,853
  2. 2021HuBERT: Self-Supervised Speech Representation Learning by Masked Prediction of Hidden Unitscited by 2,301
  3. 2021WavLM: Large-Scale Self-Supervised Pre-Training for Full Stack Speech Processingcited by 1,609
  4. 2019Common Voice: A Massively-Multilingual Speech Corpuscited by 1,358
  5. 2021SUPERB: Speech processing Universal PERformance Benchmarkcited by 652
  6. 2021XLS-R: Self-supervised Cross-lingual Speech Representation Learning at Scalecited by 589
  7. 2020Unsupervised Cross-lingual Representation Learning for Speech Recognitioncited by 496
  8. 2021VoxPopuli: A Large-Scale Multilingual Speech Corpus for Representation Learning, Semi-Supervised Learning and Interpretationcited by 455
  9. 2023Scaling Speech Technology to 1,000+ Languagescited by 401
  10. 2022Self-Supervised Speech Representation Learning: A Reviewcited by 258
  11. 2021Textless Speech-to-Speech Translation on Real Datacited by 121
  12. 2020Massively Multilingual ASR: 50 Languages, 1 Model, 1 Billion Parameterscited by 110
26 more
  1. 2023ML-SUPERB: Multilingual Speech Universal PERformance Benchmarkcited by 70
  2. 2021Scaling End-to-End Models for Large-Scale Multilingual ASRcited by 61
  3. 2021Multi-Singer: Fast Multi-Singer Singing Voice Vocoder With A Large-Scale Corpuscited by 60
  4. 2021ESPnet-SLU: Advancing Spoken Language Understanding through ESPnetcited by 53
  5. 2022Opencpop: A High-Quality Open Source Chinese Popular Song Corpus for Singing Voice Synthesiscited by 53
  6. 2021LeBenchmark: A Reproducible Framework for Assessing Self-Supervised Representation Learning from Speechcited by 46
  7. 2020Learning Robust and Multilingual Speech Representationscited by 39
  8. 2022IndicSUPERB: A Speech Processing Universal Performance Benchmark for Indian languagescited by 38
  9. 2022NusaCrowd: Open Source Initiative for Indonesian NLP Resourcescited by 38
  10. 2022SpeechMatrix: A Large-Scale Mined Corpus of Multilingual Speech-to-Speech Translationscited by 35
  11. 2023Improving Massively Multilingual ASR With Auxiliary CTC Objectivescited by 31
  12. 2022ASR2K: Speech Recognition for Around 2000 Languages without Audiocited by 26
  13. 2022SUPERB @ SLT 2022: Challenge on Generalization and Efficiency of Self-Supervised Speech Representation Learningcited by 26
  14. 2022XTREME-S: Evaluating Cross-lingual Speech Representationscited by 22
  15. 2020Self-Supervised Representations Improve End-to-End Speech Translationcited by 21
  16. 2023Reducing Barriers to Self-Supervised Learning: HuBERT Pre-training with Academic Computecited by 21
  17. 2021Improved Language Identification Through Cross-Lingual Self-Supervised Learningcited by 20
  18. 2021Leveraging End-to-End ASR for Endangered Language Documentation: An Empirical Study on Yolox'ochitl Mixteccited by 15
  19. 2022Combining Spectral and Self-Supervised Features for Low Resource Speech Recognition and Translationcited by 11
  20. 2022Muskits: an End-to-End Music Processing Toolkit for Singing Voice Synthesiscited by 10
  21. 2023Joint Prediction and Denoising for Large-scale Multilingual Self-supervised Learningcited by 8
  22. 2023Exploration on HuBERT with Multiple Resolutionscited by 7
  23. 2023`Ir`oy`inSpeech: A multi-purpose Yor`ub'a Speech Corpuscited by 7
  24. 2023Evaluating Self-Supervised Speech Representations for Indigenous American Languagescited by 6
  25. 2022FeaRLESS: Feature Refinement Loss for Ensembling Self-Supervised Learning Features in Robust End-to-end Speech Recognitioncited by 5
  26. 2023SSHR: Leveraging Self-supervised Hierarchical Representations for Multilingual Automatic Speech Recognitioncited by 4

Built on by

Cited by year

Citing papers per year: 2024: 7; 2025: 3.

10 papers.

  1. 2024mHuBERT-147: A Compact Multilingual HuBERT Modelcited by 64
  2. 2024Towards Robust Speech Representation Learning for Thousands of Languagescited by 48
  3. 2024ML-SUPERB 2.0: Benchmarking Multilingual Speech Models Across Modeling Constraints, Languages, and Datasetscited by 26
  4. 2024Wav2Gloss: Generating Interlinear Glossed Text from Speechcited by 8
  5. 2025The ML-SUPERB 2.0 Challenge: Towards Inclusive ASR Benchmarking for All Language Varietiescited by 4
  6. 2024CA-SSLR: Condition-Aware Self-Supervised Learning Representation for Generalized Speech Processingcited by 2
  7. 2025Speech-FT: Merging Pre-trained And Fine-Tuned Speech Representation Models For Cross-Task Generalizationcited by 2
  8. 2025An Exploration of Mamba for Speech Self-Supervised Modelscited by 2
  9. 2024The Faetar Benchmark: Speech Recognition in a Very Under-Resourced Languagecited by 0
  10. 2024ESPnet-EZ: Python-only ESPnet for Easy Fine-tuning and Integrationcited by 0