mlsanityData to 14 Oct 2026

ML-SUPERB: Multilingual Speech Universal PERformance Benchmark

arXiv 2305.10615 ↗First posted 18 May 2023cs.SD cs.CL eess.AScited by 71

Not on the map. Why?

Abstract (arXiv)

Speech processing Universal PERformance Benchmark (SUPERB) is a leaderboard to benchmark the performance of Self-Supervised Learning (SSL) models on various speech processing tasks. However, SUPERB largely considers English speech in its evaluation. This paper presents multilingual SUPERB (ML-SUPERB), covering 143 languages (ranging from high-resource to endangered), and considering both automatic speech recognition and language identification. Following the concept of SUPERB, ML-SUPERB utilizes frozen SSL features and employs a simple framework for multilingual tasks by learning a shallow downstream model. Similar to the SUPERB benchmark, we find speech SSL models can significantly improve performance compared to FBANK features. Furthermore, we find that multilingual models do not always perform better than their monolingual counterparts. We will release ML-SUPERB as a challenge with organized datasets and reproducible training scripts for future multilingual representation research.

Key facts. Cited by 71 papers in the map (data through 14 Oct 2026). Builds on 29 of its 48 references that are in the map. First posted on arXiv 18 May 2023.

Builds on

29 of 48 references.

References · 29

  1. 2020wav2vec 2.0: A Framework for Self-Supervised Learning of Speech Representationscited by 3,905
  2. 2021HuBERT: Self-Supervised Speech Representation Learning by Masked Prediction of Hidden Unitscited by 2,342
  3. 2021WavLM: Large-Scale Self-Supervised Pre-Training for Full Stack Speech Processingcited by 1,657
  4. 2019Common Voice: A Massively-Multilingual Speech Corpuscited by 1,384
  5. 2021SUPERB: Speech processing Universal PERformance Benchmarkcited by 666
  6. 2021XLS-R: Self-supervised Cross-lingual Speech Representation Learning at Scalecited by 601
  7. 2018ESPnet: End-to-End Speech Processing Toolkitcited by 558
  8. 2020Unsupervised Cross-lingual Representation Learning for Speech Recognitioncited by 501
  9. 2020MLS: A Large-Scale Multilingual Dataset for Speech Researchcited by 476
  10. 2022FLEURS: Few-shot Learning Evaluation of Universal Representations of Speechcited by 469
  11. 2021VoxPopuli: A Large-Scale Multilingual Speech Corpus for Representation Learning, Semi-Supervised Learning and Interpretationcited by 463
  12. 2022Self-Supervised Speech Representation Learning: A Reviewcited by 260
17 more
  1. 2022Comparative layer-wise analysis of self-supervised speech modelscited by 158
  2. 2021Robust wav2vec 2.0: Analyzing Domain Shift in Self-Supervised Pre-Trainingcited by 151
  3. 2021Textless Speech-to-Speech Translation on Real Datacited by 121
  4. 2022SUPERB-SG: Enhanced Speech processing Universal PERformance Benchmark for Semantic and Generative Capabilitiescited by 77
  5. 2021LeBenchmark: A Reproducible Framework for Assessing Self-Supervised Representation Learning from Speechcited by 47
  6. 2022IndicSUPERB: A Speech Processing Universal Performance Benchmark for Indian languagescited by 40
  7. 2021An Exploration of Self-Supervised Pretrained Representations for End-to-End Speech Recognitioncited by 39
  8. 2022SpeechMatrix: A Large-Scale Mined Corpus of Multilingual Speech-to-Speech Translationscited by 35
  9. 2023Improving Massively Multilingual ASR With Auxiliary CTC Objectivescited by 31
  10. 2022ASR2K: Speech Recognition for Around 2000 Languages without Audiocited by 26
  11. 2022SUPERB @ SLT 2022: Challenge on Generalization and Efficiency of Self-Supervised Speech Representation Learningcited by 26
  12. 2022Streaming End-to-End Multilingual Speech Recognition with Joint Language Identificationcited by 24
  13. 2022XTREME-S: Evaluating Cross-lingual Speech Representationscited by 22
  14. 2020Self-Supervised Representations Improve End-to-End Speech Translationcited by 21
  15. 2021Leveraging End-to-End ASR for Endangered Language Documentation: An Empirical Study on Yolox'ochitl Mixteccited by 15
  16. 2022Combining Spectral and Self-Supervised Features for Low Resource Speech Recognition and Translationcited by 11
  17. 2022Avoid Overthinking in Self-Supervised Models for Speech Recognitioncited by 5

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2023: 13; 2024: 25; 2025: 21; 2026: 12.

71 papers.

Sort:

By area:

  1. 2023Dynamic-SUPERB: Towards A Dynamic, Collaborative, and Comprehensive Instruction-Tuning Benchmark for Speechcited by 101
  2. 2025Full-Duplex-Bench: A Benchmark to Evaluate Full-duplex Spoken Dialogue Models on Turn-taking Capabilitiescited by 100
  3. 2024Self-Supervised Speech Representations are More Phonetic than Semanticcited by 71
  4. 2024mHuBERT-147: A Compact Multilingual HuBERT Modelcited by 66
  5. 2023Exploring Speech Recognition, Translation, and Understanding with Discrete Speech Units: A Comparative Studycited by 42
  6. 2024A Large-Scale Evaluation of Speech Foundation Modelscited by 40
  7. 2024ML-SUPERB 2.0: Benchmarking Multilingual Speech Models Across Modeling Constraints, Languages, and Datasetscited by 27
  8. 2024BESTOW: Efficient and Streamable Speech Language Model with the Best of Two Worlds in GPT and T5cited by 19
  9. 2023LeBenchmark 2.0: a Standardized, Replicable and Enhanced Framework for Self-supervised Representations of French Speechcited by 18
  10. 2024AfriHuBERT: A self-supervised speech representation model for African languagescited by 18
  11. 2025AudioLens: A Closer Look at Auditory Attribute Perception of Large Audio-Language Modelscited by 18
  12. 2023CL-MASR: A Continual Learning Benchmark for Multilingual ASRcited by 17
38 more
  1. 2023Zero Resource Code-switched Speech Benchmark Using Speech Utterance Pairs For Multiple Spoken Languagescited by 12
  2. 2025CS-FLEURS: A Massively Multilingual and Code-Switched Speech Datasetcited by 12
  3. 2023Findings of the 2023 ML-SUPERB Challenge: Pre-Training and Evaluation over More Languages and Beyondcited by 10
  4. 2023CoLLD: Contrastive Layer-to-layer Distillation for Compressing Multilingual Pre-trained Speech Encoderscited by 9
  5. 2023Joint Prediction and Denoising for Large-scale Multilingual Self-supervised Learningcited by 8
  6. 2024Codec-ASR: Training Performant Automatic Speech Recognition Systems with Discrete Speech Representationscited by 8
  7. 2023Evaluating Self-supervised Speech Models on a Taiwanese Hokkien Corpuscited by 7
  8. 2024Performance Analysis of Speech Encoders for Low-Resource SLU and ASR in Tunisian Dialectcited by 7
  9. 2023Evaluating Self-Supervised Speech Representations for Indigenous American Languagescited by 6
  10. 2024A Layer-Anchoring Strategy for Enhancing Cross-Lingual Speech Emotion Recognitioncited by 6
  11. 2025SENSE models: an open source solution for multilingual and multimodal semantic-based taskscited by 6
  12. 2024Interface Design for Self-Supervised Speech Modelscited by 5
  13. 2024How to Learn a New Language? An Efficient Solution for Self-Supervised Learning Models Unseen Languages Adaption in Low-Resource Scenariocited by 5
  14. 2023Decoding Emotions: A comprehensive Multilingual Study of Speech Models for Speech Emotion Recognitioncited by 4
  15. 2023SSHR: Leveraging Self-supervised Hierarchical Representations for Multilingual Automatic Speech Recognitioncited by 4
  16. 2024STAB: Speech Tokenizer Assessment Benchmarkcited by 4
  17. 2024SMILE: Speech Meta In-Context Learning for Low-Resource Language Automatic Speech Recognitioncited by 4
  18. 2024MERaLiON-SpeechEncoder: Towards a Speech Foundation Model for Singapore and Beyondcited by 4
  19. 2024Enhancing Multilingual ASR for Unseen Languages via Language Embedding Modelingcited by 4
  20. 2025mWhisper-Flamingo for Multilingual Audio-Visual Noise-Robust Speech Recognitioncited by 4
  21. 2025TS-SUPERB: A Target Speech Processing Benchmark for Speech Self-Supervised Learning Modelscited by 4
  22. 2025On-device Streaming Discrete Speech Unitscited by 4
  23. 2025The ML-SUPERB 2.0 Challenge: Towards Inclusive ASR Benchmarking for All Language Varietiescited by 4
  24. 2024Exploiting Longitudinal Speech Sessions via Voice Assistant Systems for Early Detection of Cognitive Declinecited by 3
  25. 2025Discrete Speech Unit Extraction via Independent Component Analysiscited by 3
  26. 2024Efficient Compression of Multitask Multilingual Speech Modelscited by 2
  27. 2024Improving Multilingual ASR in the Wild Using Simple N-best Re-rankingcited by 2
  28. 2024CA-SSLR: Condition-Aware Self-Supervised Learning Representation for Generalized Speech Processingcited by 2
  29. 2025Speech-FT: Merging Pre-trained And Fine-Tuned Speech Representation Models For Cross-Task Generalizationcited by 2
  30. 2025A correlation-permutation approach for speech-music encoders model mergingcited by 2
  31. 2025An Exploration of Mamba for Speech Self-Supervised Modelscited by 2
  32. 2025CAM~OES: A Comprehensive Automatic Speech Recognition Benchmark for European Portuguesecited by 2
  33. 2024Are Paralinguistic Representations all that is needed for Speech Emotion Recognition?cited by 1
  34. 2024AnnoTheia: A Semi-Automatic Annotation Toolkit for Audio-Visual Speech Technologiescited by 1
  35. 2024The X-LANCE Technical Report for Interspeech 2024 Speech Processing Using Discrete Speech Unit Challengecited by 1
  36. 2025Improving Multilingual Speech Models on ML-SUPERB 2.0: Fine-tuning with Data Augmentation and LID-Aware CTCcited by 1
  37. 2025Developing a High-performance Framework for Speech Emotion Recognition in Naturalistic Conditions Challenge for Emotional Attribute Predictioncited by 1
  38. 2025HARNESS: Lightweight Distilled Arabic Speech Foundation Modelscited by 1

50 most cited of 71 All 71 (JSON)