mlsanityData to 7 Oct 2026

ML-SUPERB 2.0: Benchmarking Multilingual Speech Models Across Modeling Constraints, Languages, and Datasets

arXiv 2406.08641 ↗First posted 12 Jun 2024cs.SD cs.CL eess.AScited by 26

Not on the map. Why?

Abstract (arXiv)

ML-SUPERB evaluates self-supervised learning (SSL) models on the tasks of language identification and automatic speech recognition (ASR). This benchmark treats the models as feature extractors and uses a single shallow downstream model, which can be fine-tuned for a downstream task. However, real-world use cases may require different configurations. This paper presents ML-SUPERB~2.0, which is a new benchmark for evaluating pre-trained SSL and supervised speech models across downstream models, fine-tuning setups, and efficient model adaptation approaches. We find performance improvements over the setup of ML-SUPERB. However, performance depends on the downstream model design. Also, we find large performance differences between languages and datasets, suggesting the need for more targeted approaches to improve multilingual ASR performance.

Key facts. Cited by 26 papers in the map (data through 7 Oct 2026). Builds on 41 of its 49 references that are in the map. First posted on arXiv 12 Jun 2024.

Builds on

41 of 49 references.

References · 41

  1. 2017Attention Is All You Needcited by 59,932
  2. 2021LoRA: Low-Rank Adaptation of Large Language Modelscited by 14,084
  3. 2019Parameter-Efficient Transfer Learning for NLPcited by 4,195
  4. 2022Robust Speech Recognition via Large-Scale Weak Supervisioncited by 3,972
  5. 2021Prefix-Tuning: Optimizing Continuous Prompts for Generationcited by 3,878
  6. 2021The Power of Scale for Parameter-Efficient Prompt Tuningcited by 3,769
  7. 2020Conformer: Convolution-augmented Transformer for Speech Recognitioncited by 1,838
  8. 2019Common Voice: A Massively-Multilingual Speech Corpuscited by 1,358
  9. 2021SUPERB: Speech processing Universal PERformance Benchmarkcited by 652
  10. 2021XLS-R: Self-supervised Cross-lingual Speech Representation Learning at Scalecited by 589
  11. 2018ESPnet: End-to-End Speech Processing Toolkitcited by 554
  12. 2022FLEURS: Few-shot Learning Evaluation of Universal Representations of Speechcited by 458
29 more
  1. 2023Scaling Speech Technology to 1,000+ Languagescited by 401
  2. 2018Fairness Without Demographics in Repeated Loss Minimizationcited by 363
  3. 2023Google USM: Scaling Automatic Speech Recognition Beyond 100 Languagescited by 228
  4. 2019A Comparative Study on Transformer vs RNN in Speech Applicationscited by 210
  5. 2020Recent Developments on ESPnet Toolkit Boosted by Conformercited by 132
  6. 2020Massively Multilingual ASR: 50 Languages, 1 Model, 1 Billion Parameterscited by 110
  7. 2022E-Branchformer: Branchformer with Enhanced merging for speech recognitioncited by 90
  8. 2024OWSM v3.1: Better and Faster Open Whisper-Style Speech Models based on E-Branchformercited by 74
  9. 2023ML-SUPERB: Multilingual Speech Universal PERformance Benchmarkcited by 70
  10. 2023Reproducing Whisper-Style Training Using an Open-Source Toolkit and Publicly Available Datacited by 42
  11. 2021An Exploration of Self-Supervised Pretrained Representations for End-to-End Speech Recognitioncited by 39
  12. 2022IndicSUPERB: A Speech Processing Universal Performance Benchmark for Indian languagescited by 38
  13. 2022NusaCrowd: Open Source Initiative for Indonesian NLP Resourcescited by 38
  14. 2021Exploiting Adapters for Cross-lingual Low-resource Speech Recognitioncited by 33
  15. 2023Improving Massively Multilingual ASR With Auxiliary CTC Objectivescited by 31
  16. 2023Speech Self-Supervised Representation Benchmarking: Are We Doing it Right?cited by 28
  17. 2022ASR2K: Speech Recognition for Around 2000 Languages without Audiocited by 26
  18. 2022Exploring Efficient-tuning Methods in Self-supervised Speech Modelscited by 26
  19. 2017Transfer Learning for Speech Recognition on a Budgetcited by 25
  20. 2022XTREME-S: Evaluating Cross-lingual Speech Representationscited by 22
  21. 2023Comparison of Multilingual Self-Supervised and Weakly-Supervised Speech Pre-Training for Adaptation to Unseen Languagescited by 20
  22. 2023CL-MASR: A Continual Learning Benchmark for Multilingual ASRcited by 16
  23. 2023A Comparative Study on E-Branchformer vs Conformer in Speech Recognition, Translation, and Understanding Taskscited by 13
  24. 2023Findings of the 2023 ML-SUPERB Challenge: Pre-Training and Evaluation over More Languages and Beyondcited by 10
  25. 2022Adaptive multilingual speech recognition with pretrained modelscited by 9
  26. 2023Joint Prediction and Denoising for Large-scale Multilingual Self-supervised Learningcited by 8
  27. 2023Evaluating Self-supervised Speech Models on a Taiwanese Hokkien Corpuscited by 7
  28. 2023Evaluating Self-Supervised Speech Representations for Indigenous American Languagescited by 6
  29. 2023SSHR: Leveraging Self-supervised Hierarchical Representations for Multilingual Automatic Speech Recognitioncited by 4

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2024: 2; 2025: 13; 2026: 11.

26 papers.

Sort:

By area:

  1. 2025BLAB: Brutally Long Audio Benchcited by 11
  2. 2025mSTEB: Massively Multilingual Evaluation of LLMs on Speech and Text Taskscited by 7
  3. 2024Exploring SSL Discrete Tokens for Multilingual ASRcited by 6
  4. 2025Benchmarking Training Paradigms, Dataset Composition, and Model Scaling for Child ASR in ESPnetcited by 6
  5. 2025Can we train ASR systems on Code-switch without real code-switch data? Case study for Singapore's languagescited by 5
  6. 2025The Eloquence team submission for task 1 of MLC-SLM challengecited by 4
  7. 2025Crossing the Species Divide: Transfer Learning from Speech to Animal Soundscited by 4
  8. 2025The ML-SUPERB 2.0 Challenge: Towards Inclusive ASR Benchmarking for All Language Varietiescited by 4
  9. 2025CTC-DRO: Robust Optimization for Reducing Language Disparities in Speech Recognitioncited by 1
  10. 2025Improving Multilingual Speech Models on ML-SUPERB 2.0: Fine-tuning with Data Augmentation and LID-Aware CTCcited by 1
  11. 2025Developing a High-performance Framework for Speech Emotion Recognition in Naturalistic Conditions Challenge for Emotional Attribute Predictioncited by 1
  12. 2025SHNU Multilingual Conversational Speech Recognition System for INTERSPEECH 2025 MLC-SLM Challengecited by 1
14 more
  1. 2025MLMA: Towards Multilingual ASR With Mamba-based Architecturescited by 1
  2. 2026Multimodal In-context Learning for ASR of Low-resource Languagescited by 1
  3. 2026Do Neural Codecs Generalize? A Controlled Study Across Unseen Languages and Non-Speech Taskscited by 1
  4. 2026Adapting Where It Matters: Depth-Aware Adaptation for Efficient Multilingual Speech Recognition in Low-Resource Languagescited by 1
  5. 2026LoASR-Bench: Evaluating Large Speech Language Models on Low-Resource Automatic Speech Recognition Across Language Familiescited by 1
  6. 2026Multi-layer attentive probing improves transfer of audio representations for bioacousticscited by 1
  7. 2026Afrispeech Semantics: Evaluating Audio Semantic Reasoning in Spoken Language Models Across Domains and Accentscited by 1
  8. 2026Gumbel-BEARD: Automatic Layer Selection for Self-Supervised Adaptation of Whisper in Low-Resource Domainscited by 1
  9. 2024ESPnet-EZ: Python-only ESPnet for Easy Fine-tuning and Integrationcited by 0
  10. 2025An Entropy-Guided Curriculum Learning Strategy for Data-Efficient Acoustic Scene Classification under Domain Shiftcited by 0
  11. 2026Continual Adaptation for Pacific Indigenous Speech Recognitioncited by 0
  12. 2026MUSCAT: MUltilingual, SCientific ConversATion Benchmarkcited by 0
  13. 2026MoLGE: Mixture of Language Group Experts for Efficient Scaling of Massively Multilingual Speech Recognitioncited by 0
  14. 2026Turning Speech Language Models into Multilingual Listenerscited by 0