mlsanityData to 7 Oct 2026

Unsupervised Cross-lingual Representation Learning for Speech Recognition

arXiv 2006.13979 ↗First posted 24 Jun 2020cs.CL cs.LG cs.SDcited by 496

Not on the map. Why?

Abstract (arXiv)

This paper presents XLSR which learns cross-lingual speech representations by pretraining a single model from the raw waveform of speech in multiple languages. We build on wav2vec 2.0 which is trained by solving a contrastive task over masked latent speech representations and jointly learns a quantization of the latents shared across languages. The resulting model is fine-tuned on labeled data and experiments show that cross-lingual pretraining significantly outperforms monolingual pretraining. On the CommonVoice benchmark, XLSR shows a relative phoneme error rate reduction of 72% compared to the best known results. On BABEL, our approach improves word error rate by 16% relative compared to a comparable system. Our approach enables a single multilingual speech recognition model which is competitive to strong individual models. Analysis shows that the latent discrete speech representations are shared across languages with increased sharing for related languages. We hope to catalyze research in low-resource speech understanding by releasing XLSR-53, a large model pretrained in 53 languages.

Key facts. Cited by 496 papers in the map (data through 7 Oct 2026). First posted on arXiv 24 Jun 2020.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2021: 57; 2022: 87; 2023: 109; 2024: 108; 2025: 71; 2026: 64.

496 papers.

Sort:

By area:

  1. 2021FLAVA: A Foundational Language And Vision Alignment Modelcited by 646
  2. 2021XLS-R: Self-supervised Cross-lingual Speech Representation Learning at Scalecited by 589
  3. 2022FLEURS: Few-shot Learning Evaluation of Universal Representations of Speechcited by 458
  4. 2021VoxPopuli: A Large-Scale Multilingual Speech Corpus for Representation Learning, Semi-Supervised Learning and Interpretationcited by 455
  5. 2023Scaling Speech Technology to 1,000+ Languagescited by 401
  6. 2021Layer-wise Analysis of a Self-supervised Speech Representation Modelcited by 293
  7. 2022Self-Supervised Speech Representation Learning: A Reviewcited by 258
  8. 2023A Survey on Deep Neural Network Pruning-Taxonomy, Comparison, Analysis, and Recommendationscited by 200
  9. 2021WenetSpeech: A 10000+ Hours Multi-domain Mandarin Corpus for Speech Recognitioncited by 189
  10. 2022Self-supervised Learning with Random-projection Quantizer for Speech Recognitioncited by 158
  11. 2022Comparative layer-wise analysis of self-supervised speech modelscited by 155
  12. 2021Robust wav2vec 2.0: Analyzing Domain Shift in Self-Supervised Pre-Trainingcited by 148
38 more
  1. 2021Unsupervised Speech Recognitioncited by 147
  2. 2023Prompting Large Language Models with Speech Recognition Abilitiescited by 139
  3. 2022mSLAM: Massively multilingual joint pre-training for speech and textcited by 108
  4. 2023A Complete Survey on Generative AI (AIGC): Is ChatGPT from GPT-4 to GPT-5 All You Need?cited by 84
  5. 2021Neural Analysis and Synthesis: Reconstructing Speech from Self-Supervised Representationscited by 81
  6. 2021Simple and Effective Zero-shot Cross-lingual Phoneme Recognitioncited by 79
  7. 2022FreeVC: Towards High-Quality Text-Free One-Shot Voice Conversioncited by 79
  8. 2021UniSpeech: Unified Speech Representation Learning with Labeled and Unlabeled Datacited by 76
  9. 2023ML-SUPERB: Multilingual Speech Universal PERformance Benchmarkcited by 70
  10. 2023A Comprehensive Survey on Applications of Transformers for Deep Learning Taskscited by 70
  11. 2021SLAM: A Unified Encoder for Speech and Language Modeling via Speech-Text Joint Pre-Trainingcited by 69
  12. 2021Performance-Efficiency Trade-offs in Unsupervised Pre-training for Speech Recognitioncited by 68
  13. 2023SelfTalk: A Self-Supervised Commutative Training Diagram to Comprehend 3D Talking Facescited by 67
  14. 2024mHuBERT-147: A Compact Multilingual HuBERT Modelcited by 64
  15. 2021Scaling End-to-End Models for Large-Scale Multilingual ASRcited by 61
  16. 2021ESPnet-SLU: Advancing Spoken Language Understanding through ESPnetcited by 53
  17. 2023Introduction to Latent Variable Energy-Based Models: A Path Towards Autonomous Machine Intelligencecited by 52
  18. 2021PARP: Prune, Adjust and Re-Prune for Self-Supervised Speech Recognitioncited by 50
  19. 2022ByT5 model for massively multilingual grapheme-to-phoneme conversioncited by 49
  20. 2024Towards Robust Speech Representation Learning for Thousands of Languagescited by 48
  21. 2021LeBenchmark: A Reproducible Framework for Assessing Self-Supervised Representation Learning from Speechcited by 46
  22. 2021Towards Building ASR Systems for the Next Billion Userscited by 45
  23. 2022Towards End-to-end Unsupervised Speech Recognitioncited by 44
  24. 2021CDPAM: Contrastive learning for perceptual audio similaritycited by 42
  25. 2025Recent Advances in Discrete Speech Tokens: A Reviewcited by 42
  26. 2024SEACrowd: A Multilingual Multimodal Data Hub and Benchmark Suite for Southeast Asian Languagescited by 41
  27. 2021Joint Unsupervised and Supervised Training for Multilingual ASRcited by 39
  28. 2021Large-Scale Self- and Semi-Supervised Learning for Speech Translationcited by 36
  29. 2022u-HuBERT: Unified Mixed-Modal Speech Pretraining And Zero-Shot Transfer to Unlabeled Modalitycited by 36
  30. 2021Multilingual and Cross-Lingual Intent Detection from Spoken Datacited by 35
  31. 2023DinoSR: Self-Distillation and Online Clustering for Self-supervised Speech Representation Learningcited by 34
  32. 2023AfriSpeech-200: Pan-African Accented Speech Dataset for Clinical and General Domain ASRcited by 34
  33. 2024GigaSpeech 2: An Evolving, Large-Scale and Multi-domain ASR Corpus for Low-Resource Languages with Automated Crawling, Transcription and Refinementcited by 34
  34. 2021Exploiting Adapters for Cross-lingual Low-resource Speech Recognitioncited by 33
  35. 2023Quantifying the Dialect Gap and its Correlates Across Languagescited by 33
  36. 2022SAMU-XLSR: Semantically-Aligned Multimodal Utterance-level Cross-Lingual Speech Representationcited by 32
  37. 2023Improving Massively Multilingual ASR With Auxiliary CTC Objectivescited by 31
  38. 2024UniTalker: Scaling up Audio-Driven 3D Facial Animation through A Unified Modelcited by 29

50 most cited of 496 All 496 (JSON)