mlsanityData to 7 Oct 2026

XTREME-S: Evaluating Cross-lingual Speech Representations

arXiv 2203.10752 ↗First posted 21 Mar 2022cs.CLcited by 22

Not on the map. Why?

Abstract (arXiv)

We introduce XTREME-S, a new benchmark to evaluate universal cross-lingual speech representations in many languages. XTREME-S covers four task families: speech recognition, classification, speech-to-text translation and retrieval. Covering 102 languages from 10+ language families, 3 different domains and 4 task families, XTREME-S aims to simplify multilingual speech representation evaluation, as well as catalyze research in "universal" speech representation learning. This paper describes the new benchmark and establishes the first speech-only and speech-text baselines using XLS-R and mSLAM on all downstream tasks. We motivate the design choices and detail how to use the benchmark. Datasets and fine-tuning scripts are made easily accessible at https://hf.co/datasets/google/xtreme_s.

Key facts. Cited by 22 papers in the map (data through 7 Oct 2026). Builds on 32 of its 41 references that are in the map. First posted on arXiv 21 Mar 2022.

Builds on

32 of 41 references.

References · 32

  1. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,204
  2. 2019Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformercited by 14,491
  3. 2018GLUE: A Multi-Task Benchmark and Analysis Platform for Natural Language Understandingcited by 4,854
  4. 2019Unsupervised Cross-lingual Representation Learning at Scalecited by 3,907
  5. 2020wav2vec 2.0: A Framework for Self-Supervised Learning of Speech Representationscited by 3,853
  6. 2019XLNet: Generalized Autoregressive Pretraining for Language Understandingcited by 2,801
  7. 2019SuperGLUE: A Stickier Benchmark for General-Purpose Language Understanding Systemscited by 1,718
  8. 2019Common Voice: A Massively-Multilingual Speech Corpuscited by 1,358
  9. 2018XNLI: Evaluating Cross-lingual Sentence Representationscited by 913
  10. 2020XTREME: A Massively Multilingual Multi-task Benchmark for Evaluating Cross-lingual Generalizationcited by 738
  11. 2021SUPERB: Speech processing Universal PERformance Benchmarkcited by 652
  12. 2021XLS-R: Self-supervised Cross-lingual Speech Representation Learning at Scalecited by 589
20 more
  1. 2021The FLORES-101 Evaluation Benchmark for Low-Resource and Multilingual Machine Translationcited by 556
  2. 2020Unsupervised Cross-lingual Representation Learning for Speech Recognitioncited by 496
  3. 2020TyDi QA: A Benchmark for Information-Seeking Question Answering in Typologically Diverse Languagescited by 486
  4. 2020MLS: A Large-Scale Multilingual Dataset for Speech Researchcited by 467
  5. 2021VoxPopuli: A Large-Scale Multilingual Speech Corpus for Representation Learning, Semi-Supervised Learning and Interpretationcited by 455
  6. 2018Massively Multilingual Sentence Embeddings for Zero-Shot Cross-Lingual Transfer and Beyondcited by 434
  7. 2021W2v-BERT: Combining Contrastive Learning and Masked Language Modeling for Self-Supervised Speech Pre-Trainingcited by 340
  8. 2019MLQA: Evaluating Cross-lingual Extractive Question Answeringcited by 339
  9. 2021XTREME-R: Towards More Challenging and Nuanced Multilingual Evaluationcited by 167
  10. 2021Unsupervised Speech Recognitioncited by 147
  11. 2020VoxLingua107: a Dataset for Spoken Language Recognitioncited by 118
  12. 2019Europarl-ST: A Multilingual Corpus For Speech Translation Of Parliamentary Debatescited by 114
  13. 2022mSLAM: Massively multilingual joint pre-training for speech and textcited by 108
  14. 2021The Multilingual TEDx Corpus for Speech Recognition and Translationcited by 101
  15. 2020Unsupervised pretraining transfers well across languagescited by 100
  16. 2020Self-training and Pre-training are Complementary for Speech Recognitioncited by 98
  17. 2022CVSS Corpus and Massively Multilingual Speech-to-Speech Translationcited by 71
  18. 2021Multilingual and Cross-Lingual Intent Detection from Spoken Datacited by 35
  19. 2018Confidence Estimation and Deletion Prediction Using Bidirectional Recurrent Neural Networkscited by 12
  20. 2018Transfer learning of language-independent end-to-end ASR with language model fusioncited by 10

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2022: 5; 2023: 7; 2024: 5; 2025: 2; 2026: 3.

22 papers.

Sort:

  1. 2022AudioLM: a Language Modeling Approach to Audio Generationcited by 611
  2. 2023ML-SUPERB: Multilingual Speech Universal PERformance Benchmarkcited by 70
  3. 2024SEACrowd: A Multilingual Multimodal Data Hub and Benchmark Suite for Southeast Asian Languagescited by 41
  4. 2024A Large-Scale Evaluation of Speech Foundation Modelscited by 38
  5. 2024ML-SUPERB 2.0: Benchmarking Multilingual Speech Models Across Modeling Constraints, Languages, and Datasetscited by 26
  6. 2023Can ChatGPT Detect Intent? Evaluating Large Language Models for Spoken Language Understandingcited by 21
  7. 2022ESB: A Benchmark For Multi-Domain End-to-End Speech Recognitioncited by 18
  8. 2024BESTOW: Efficient and Streamable Speech Language Model with the Best of Two Worlds in GPT and T5cited by 18
  9. 2023CL-MASR: A Continual Learning Benchmark for Multilingual ASRcited by 16
  10. 2023Findings of the 2023 ML-SUPERB Challenge: Pre-Training and Evaluation over More Languages and Beyondcited by 10
  11. 2024State of NLP in Kenya: A Surveycited by 7
  12. 2022ON-TRAC Consortium Systems for the IWSLT 2022 Dialect and Low-resource Speech Translation Taskscited by 6
10 more
  1. 2022A Compact End-to-End Model with Local and Global Context for Spoken Language Identificationcited by 4
  2. 2022T5lephone: Bridging Speech and Text Self-supervised Models for Spoken Language Understanding via Phoneme level T5cited by 4
  3. 2023The Interpreter Understands Your Meaning: End-to-end Spoken Language Understanding Aided by Speech Translationcited by 4
  4. 2023bgGLUE: A Bulgarian General Language Understanding Evaluation Benchmarkcited by 4
  5. 2025TS-SUPERB: A Target Speech Processing Benchmark for Speech Self-Supervised Learning Modelscited by 3
  6. 2026GlobeAudio: A Multilingual Multicultural Benchmark for Naturalistic Evaluation of Large Audio-Language Modelscited by 2
  7. 2023HuBERTopic: Enhancing Semantic Representation of HuBERT through Self-supervision Utilizing Topic Modelcited by 0
  8. 2025WolBanking77: Wolof Banking Speech Intent Classification Datasetcited by 0
  9. 2026AfriVoices-KE: A Multilingual Speech Dataset for Kenyan Languagescited by 0
  10. 2026BlasBench: An Open Benchmark for Irish Speech Recognitioncited by 0