mlsanityData to 14 Oct 2026

Magic dust for cross-lingual adaptation of monolingual wav2vec-2.0

arXiv 2110.03560 ↗First posted 7 Oct 2021cs.CL cs.SD eess.AScited by 12

Not on the map. Why?

Abstract (arXiv)

We propose a simple and effective cross-lingual transfer learning method to adapt monolingual wav2vec-2.0 models for Automatic Speech Recognition (ASR) in resource-scarce languages. We show that a monolingual wav2vec-2.0 is a good few-shot ASR learner in several languages. We improve its performance further via several iterations of Dropout Uncertainty-Driven Self-Training (DUST) by using a moderate-sized unlabeled speech dataset in the target language. A key finding of this work is that the adapted monolingual wav2vec-2.0 achieves similar performance as the topline multilingual XLSR model, which is trained on fifty-three languages, on the target language ASR task.

Key facts. Cited by 12 papers in the map (data through 14 Oct 2026). Builds on 19 of its 27 references that are in the map. First posted on arXiv 7 Oct 2021.

Builds on

19 of 27 references.

References · 19

  1. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,399
  2. 2020A Simple Framework for Contrastive Learning of Visual Representationscited by 12,049
  3. 2018Representation Learning with Contrastive Predictive Codingcited by 7,777
  4. 2020wav2vec 2.0: A Framework for Self-Supervised Learning of Speech Representationscited by 3,905
  5. 2019fairseq: A Fast, Extensible Toolkit for Sequence Modelingcited by 1,488
  6. 2020Unsupervised Cross-lingual Representation Learning for Speech Recognitioncited by 501
  7. 2019Libri-Light: A Benchmark for ASR with Limited or No Supervisioncited by 467
  8. 2020MAD-X: An Adapter-Based Framework for Multi-Task Cross-Lingual Transfercited by 445
  9. 2019Revisiting Self-Training for Neural Sequence Generationcited by 155
  10. 2021Robust wav2vec 2.0: Analyzing Domain Shift in Self-Supervised Pre-Trainingcited by 151
  11. 2019Self-Training for End-to-End Speech Recognitioncited by 129
  12. 2020Unsupervised pretraining transfers well across languagescited by 101
7 more
  1. 2019Large-Scale Multilingual Speech Recognition with a Streaming End-to-End Modelcited by 93
  2. 2020Iterative Pseudo-Labeling for Speech Recognitioncited by 83
  3. 2020Unsupervised Domain Adaptation for Speech Recognition via Uncertainty Driven Self-Trainingcited by 39
  4. 2015Transfer Learning for Speech and Language Processingcited by 35
  5. 2021Exploiting Adapters for Cross-lingual Low-resource Speech Recognitioncited by 35
  6. 2020The 2020 ESPnet update: new features, broadened applications, performance improvements, and future planscited by 14
  7. 2019Exploiting semi-supervised training through a dropout regularization in end-to-end speech recognitioncited by 5

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2022: 4; 2023: 6; 2024: 0; 2025: 1; 2026: 1.

12 papers.

  1. 2023Making More of Little Data: Improving Low-Resource Automatic Speech Recognition Using Data Augmentationcited by 31
  2. 2023Deep Transfer Learning for Automatic Speech Recognition: Towards Better Generalizationcited by 28
  3. 2023Comparison of Multilingual Self-Supervised and Weakly-Supervised Speech Pre-Training for Adaptation to Unseen Languagescited by 20
  4. 2023Adapting Multilingual Speech Representation Model for a New, Underresourced Language through Multilingual Fine-tuning and Continued Pretrainingcited by 18
  5. 2022A Brief Overview of Unsupervised Neural Speech Representation Learningcited by 9
  6. 2022Sample-Efficient Unsupervised Domain Adaptation of Speech Recognition Systems A case study for Modern Greekcited by 7
  7. 2022Unsupervised domain adaptation for speech recognition with unsupervised error correctioncited by 5
  8. 2023Stable Distillation: Regularizing Continued Pre-training for Low-Resource Automatic Speech Recognitioncited by 2
  9. 2022On Unsupervised Uncertainty-Driven Speech Pseudo-Label Filtering and Model Calibrationcited by 1
  10. 2025On the Cross-lingual Transferability of Pre-trained wav2vec2-based Modelscited by 1
  11. 2023Federated Representation Learning for Automatic Speech Recognitioncited by 0
  12. 2026Closing the Speech-Text Gap with Limited Audio for Effective Domain Adaptation in LLM-Based ASRcited by 0