mlsanityData to 7 Oct 2026

Learning Robust and Multilingual Speech Representations

arXiv 2001.11128 ↗First posted 29 Jan 2020cs.CL cs.LG eess.AScited by 39

Not on the map. Why?

Abstract (arXiv)

Unsupervised speech representation learning has shown remarkable success at finding representations that correlate with phonetic structures and improve downstream speech recognition performance. However, most research has been focused on evaluating the representations in terms of their ability to improve the performance of speech recognition systems on read English (e.g. Wall Street Journal and LibriSpeech). This evaluation methodology overlooks two important desiderata that speech representations should have: robustness to domain shifts and transferability to other languages. In this paper we learn representations from up to 8000 hours of diverse and noisy speech data and evaluate the representations by looking at their robustness to domain shifts and their ability to improve recognition performance in many languages. We find that our representations confer significant robustness advantages to the resulting recognition systems: we see significant improvements in out-of-domain transfer relative to baseline feature sets and the features likewise provide improvements in 25 phonetically diverse languages including tonal languages and low-resource languages.

Key facts. Cited by 39 papers in the map (data through 7 Oct 2026). First posted on arXiv 29 Jan 2020.

Builds on

No reference list yet.

Built on by

Cited by year

Citing papers per year: 2021: 15; 2022: 13; 2023: 8; 2024: 2; 2025: 1.

39 papers.

Sort:

  1. 2021XLS-R: Self-supervised Cross-lingual Speech Representation Learning at Scalecited by 589
  2. 2022Self-Supervised Speech Representation Learning: A Reviewcited by 258
  3. 2023Google USM: Scaling Automatic Speech Recognition Beyond 100 Languagescited by 228
  4. 2021Robust wav2vec 2.0: Analyzing Domain Shift in Self-Supervised Pre-Trainingcited by 148
  5. 2021Unsupervised Speech Recognitioncited by 147
  6. 2022mSLAM: Massively multilingual joint pre-training for speech and textcited by 108
  7. 2021BigSSL: Exploring the Frontier of Large-Scale Semi-Supervised Learning for Automatic Speech Recognitioncited by 88
  8. 2021UniSpeech: Unified Speech Representation Learning with Labeled and Unlabeled Datacited by 76
  9. 2022Audio Self-supervised Learning: A Surveycited by 65
  10. 2021LeBenchmark: A Reproducible Framework for Assessing Self-Supervised Representation Learning from Speechcited by 46
  11. 2021Multi-Format Contrastive Learning of Audio Representationscited by 39
  12. 2023Transformers in Speech Processing: A Surveycited by 38
27 more
  1. 2022u-HuBERT: Unified Mixed-Modal Speech Pretraining And Zero-Shot Transfer to Unlabeled Modalitycited by 36
  2. 2022How Does Pre-trained Wav2Vec 2.0 Perform on Domain Shifted ASR? An Extensive Benchmark on Air Traffic Control Communicationscited by 30
  3. 2022Whodunit? Learning to Contrast for Authorship Attributioncited by 17
  4. 2021Improving the Adversarial Robustness for Speaker Verification by Self-Supervised Learningcited by 16
  5. 2022Differentially Private Speaker Anonymizationcited by 16
  6. 2022A Study of Gender Impact in Self-supervised Models for Speech-to-Text Systemscited by 14
  7. 2021Improving Noise Robustness of Contrastive Speech Representation Learning with Speech Reconstructioncited by 12
  8. 2021An Evaluation of Self-Supervised Pre-Training for Skin-Lesion Analysiscited by 10
  9. 2023Findings of the 2023 ML-SUPERB Challenge: Pre-Training and Evaluation over More Languages and Beyondcited by 10
  10. 2022A Brief Overview of Unsupervised Neural Speech Representation Learningcited by 9
  11. 2021On Architectures and Training for Raw Waveform Feature Extraction in ASRcited by 8
  12. 2021Self-Supervised Learning from Automatically Separated Sound Scenescited by 6
  13. 2022ON-TRAC Consortium Systems for the IWSLT 2022 Dialect and Low-resource Speech Translation Taskscited by 6
  14. 2023Cocktail HuBERT: Generalized Self-Supervised Pre-training for Mixture and Single-Source Speechcited by 6
  15. 2024Efficient Training of Self-Supervised Speech Foundation Models on a Compute Budgetcited by 5
  16. 2021Fast Development of ASR in African Languages using Self Supervised Speech Representation Learningcited by 3
  17. 2023Exploration of Language Dependency for Japanese Self-Supervised Speech Representation Modelscited by 3
  18. 2021Injecting Text and Cross-lingual Supervision in Few-shot Learning from Self-Supervised Modelscited by 2
  19. 2021Learning Speaker Representation with Semi-supervised Learning approach for Speaker Profilingcited by 2
  20. 2023Unsupervised Pre-Training for Vietnamese Automatic Speech Recognition in the HYKIST Projectcited by 2
  21. 2023Spatial HuBERT: Self-supervised Spatial Speech Representation Learning for a Single Talker from Multi-channel Audiocited by 2
  22. 2025Seal Your Backdoor with Variational Defensecited by 2
  23. 2023Supervised Acoustic Embeddings And Their Transferability Across Languagescited by 1
  24. 2022Experiments on Turkish ASR with Self-Supervised Speech Representation Learningcited by 0
  25. 2022Investigating self-supervised, weakly supervised and fully supervised training approaches for multi-domain automatic speech recognition: a study on Bangladeshi Banglacited by 0
  26. 2022Unsupervised Fine-Tuning Data Selection for ASR Using Self-Supervised Speech Modelscited by 0
  27. 2024Cross-Modal Coordination Across a Diverse Set of Input Modalitiescited by 0