mlsanityData to 7 Oct 2026

Unsupervised ASR via Cross-Lingual Pseudo-Labeling

arXiv 2305.13330 ↗First posted 19 May 2023eess.AS cs.CL cs.LGcited by 0

Not on the map. Why?

Abstract (arXiv)

Recent work has shown that it is possible to train an textitunsupervised automatic speech recognition (ASR) system using only unpaired audio and text. Existing unsupervised ASR methods assume that no labeled data can be used for training. We argue that even if one does not have any labeled audio for a given language, there is textitalways labeled data available for other languages. We show that it is possible to use character-level acoustic models (AMs) from other languages to bootstrap an textitunsupervised AM in a new language. Here, "unsupervised" means no labeled audio is available for the textittarget language. Our approach is based on two key ingredients: (i) generating pseudo-labels (PLs) of the textittarget language using some textitother language AM and (ii) constraining these PLs with a textittarget language model. Our approach is effective on Common Voice: e.g. transfer of English AM to Swahili achieves 18% WER. It also outperforms character-based wav2vec-U 2.0 by 15% absolute WER on LJSpeech with 800h of labeled German data instead of 60k hours of unlabeled English data.

Key facts. No citing papers in the map yet (data through 7 Oct 2026). Builds on 33 of its 45 references that are in the map. First posted on arXiv 19 May 2023.

Builds on

33 of 45 references.

References · 33

  1. 2017Attention Is All You Needcited by 59,932
  2. 2019Unsupervised Cross-lingual Representation Learning at Scalecited by 3,907
  3. 2020wav2vec 2.0: A Framework for Self-Supervised Learning of Speech Representationscited by 3,853
  4. 2021HuBERT: Self-Supervised Speech Representation Learning by Masked Prediction of Hidden Unitscited by 2,301
  5. 2019Common Voice: A Massively-Multilingual Speech Corpuscited by 1,358
  6. 2019SpecAugment: A Simple Data Augmentation Method for Automatic Speech Recognitioncited by 1,355
  7. 2022data2vec: A General Framework for Self-supervised Learning in Speech, Vision and Languagecited by 718
  8. 2019CCNet: Extracting High Quality Monolingual Datasets from Web Crawl Datacited by 544
  9. 2021W2v-BERT: Combining Contrastive Learning and Masked Language Modeling for Self-Supervised Speech Pre-Trainingcited by 340
  10. 2021Unsupervised Speech Recognitioncited by 147
  11. 2020Improved Noisy Student Training for Automatic Speech Recognitioncited by 139
  12. 2019Self-Training for End-to-End Speech Recognitioncited by 128
21 more
  1. 2020Massively Multilingual ASR: 50 Languages, 1 Model, 1 Billion Parameterscited by 110
  2. 2020Self-training and Pre-training are Complementary for Speech Recognitioncited by 98
  3. 2021BigSSL: Exploring the Frontier of Large-Scale Semi-Supervised Learning for Automatic Speech Recognitioncited by 88
  4. 2021TorchAudio: Building Blocks for Audio and Speech Processingcited by 83
  5. 2020Iterative Pseudo-Labeling for Speech Recognitioncited by 81
  6. 2020SlimIPL: Language-Model-Free Iterative Pseudo-Labelingcited by 54
  7. 2022Squeezeformer: An Efficient Transformer for Automatic Speech Recognitioncited by 50
  8. 2022Towards End-to-end Unsupervised Speech Recognitioncited by 44
  9. 2021Efficient conformer: Progressive downsampling and grouped attention for automatic speech recognitioncited by 42
  10. 2021Momentum Pseudo-Labeling for Semi-Supervised Speech Recognitioncited by 37
  11. 2020Improved Mask-CTC for Non-Autoregressive End-to-End ASRcited by 32
  12. 2022ASR2K: Speech Recognition for Around 2000 Languages without Audiocited by 26
  13. 2022Flashlight: Enabling Innovation in Tools for Machine Learningcited by 25
  14. 2021Pseudo-Labeling for Massively Multilingual Speech Recognitioncited by 23
  15. 2021Kaizen: Continuously improving teacher using Exponential Moving Average for semi-supervised speech recognitioncited by 21
  16. 2022Maestro-U: Leveraging joint speech-text representation learning for zero supervised speech ASRcited by 13
  17. 2021XLST: Cross-lingual Self-training to Learn Multilingual Representation for Low Resource Speech Recognitioncited by 11
  18. 2021Deciphering Speech: a Zero-Resource Approach to Cross-Lingual Transfer in ASRcited by 11
  19. 2022Continuous Pseudo-Labeling from the Startcited by 11
  20. 2021Unsupervised Automatic Speech Recognition: A Reviewcited by 5
  21. 2022More Speaking or More Speakers?cited by 2

Built on by

Not cited yet.