mlsanityData to 7 Oct 2026

Towards End-to-end Unsupervised Speech Recognition

arXiv 2204.02492 ↗First posted 5 Apr 2022cs.CL cs.SD eess.AScited by 44

Not on the map. Why?

Abstract (arXiv)

Unsupervised speech recognition has shown great potential to make Automatic Speech Recognition (ASR) systems accessible to every language. However, existing methods still heavily rely on hand-crafted pre-processing. Similar to the trend of making supervised speech recognition end-to-end, we introduce wav2vec-U 2.0 which does away with all audio-side pre-processing and improves accuracy through better architecture. In addition, we introduce an auxiliary self-supervised objective that ties model predictions back to the input. Experiments show that wav2vec-U 2.0 improves unsupervised recognition results across different languages while being conceptually simpler.

Key facts. Cited by 44 papers in the map (data through 7 Oct 2026). Builds on 27 of its 42 references that are in the map. First posted on arXiv 5 Apr 2022.

Builds on

27 of 42 references.

References · 27

  1. 2015Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shiftcited by 6,979
  2. 2022Generative Adversarial Networkscited by 5,603
  3. 2020wav2vec 2.0: A Framework for Self-Supervised Learning of Speech Representationscited by 3,853
  4. 2017Improved Training of Wasserstein GANscited by 2,223
  5. 2020Conformer: Convolution-augmented Transformer for Speech Recognitioncited by 1,838
  6. 2019fairseq: A Fast, Extensible Toolkit for Sequence Modelingcited by 1,486
  7. 2019Common Voice: A Massively-Multilingual Speech Corpuscited by 1,358
  8. 2019SpecAugment: A Simple Data Augmentation Method for Automatic Speech Recognitioncited by 1,355
  9. 2020Unsupervised Cross-lingual Representation Learning for Speech Recognitioncited by 496
  10. 2020MLS: A Large-Scale Multilingual Dataset for Speech Researchcited by 467
  11. 2015Deep Speech 2: End-to-End Speech Recognition in English and Mandarincited by 449
  12. 2020Pushing the Limits of Semi-Supervised Learning for Automatic Speech Recognitioncited by 185
15 more
  1. 2021Unsupervised Speech Recognitioncited by 147
  2. 2020Improved Noisy Student Training for Automatic Speech Recognitioncited by 139
  3. 2019End-to-end ASR: from Supervised to Semi-Supervised Learning with Modern Architecturescited by 120
  4. 2020Unsupervised pretraining transfers well across languagescited by 100
  5. 2020ContextNet: Improving Convolutional Neural Networks for Automatic Speech Recognition with Global Contextcited by 96
  6. 2020Iterative Pseudo-Labeling for Speech Recognitioncited by 81
  7. 2020Self-Supervised Contrastive Learning for Unsupervised Phoneme Segmentationcited by 43
  8. 2019Almost Unsupervised Text to Speech and Automatic Speech Recognitioncited by 41
  9. 2018Unsupervised Cross-Modal Alignment of Speech and Text Embedding Spacescited by 34
  10. 2019rVAD: An Unsupervised Segment-Based Robust Voice Activity Detection Methodcited by 32
  11. 2018Completely Unsupervised Phoneme Recognition by Adversarially Learning Mapping Relationships from Audio Embeddingscited by 22
  12. 2018Unsupervised Speech Recognition via Segmental Empirical Output Distribution Matchingcited by 20
  13. 2018Fully Convolutional Speech Recognitioncited by 19
  14. 2018Segmental Audio Word2Vec: Representing Utterances as Sequences of Vectors with Applications in Spoken Term Detectioncited by 17
  15. 2019Towards Unsupervised Speech Recognition and Synthesis with Quantized Speech Representation Learningcited by 16

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2022: 15; 2023: 14; 2024: 9; 2025: 4; 2026: 2.

44 papers.

Sort:

By area:

  1. 2023Scaling Speech Technology to 1,000+ Languagescited by 401
  2. 2022Self-Supervised Speech Representation Learning: A Reviewcited by 258
  3. 2024Natural Language Understanding and Inference with MLLM in Visual Question Answering: A Surveycited by 74
  4. 2024A Survey on Diffusion Models for Time Series and Spatio-Temporal Datacited by 66
  5. 2023One Small Step for Generative AI, One Giant Leap for AGI: A Complete Survey on ChatGPT in AIGC Eracited by 55
  6. 2022SpeechCLIP: Integrating Speech with Pre-Trained Vision and Language Modelcited by 29
  7. 2022SUPERB @ SLT 2022: Challenge on Generalization and Efficiency of Self-Supervised Speech Representation Learningcited by 26
  8. 2022Analyzing Acoustic Word Embeddings from Pre-trained Self-supervised Speech Modelscited by 21
  9. 2022Maestro-U: Leveraging joint speech-text representation learning for zero supervised speech ASRcited by 13
  10. 2022Simple and Effective Unsupervised Speech Synthesiscited by 12
  11. 2022Simple and Effective Unsupervised Speech Translationcited by 12
  12. 2024SpeechDPR: End-to-End Spoken Passage Retrieval for Open-Domain Spoken Question Answeringcited by 12
32 more
  1. 2023Fast-HuBERT: An Efficient Training Framework for Self-Supervised Speech Representation Learningcited by 9
  2. 2023Learning from Flawed Data: Weakly Supervised Automatic Speech Recognitioncited by 8
  3. 2022Unsupervised TTS Acoustic Modeling for TTS with Conditional Disentangled Sequential VAEcited by 7
  4. 2022On Compressing Sequences for Self-Supervised Speech Modelscited by 7
  5. 2023Pushing the Limits of Unsupervised Unit Discovery for SSL Speech Representationcited by 7
  6. 2023Improving Cascaded Unsupervised Speech Translation with Denoising Back-translationcited by 5
  7. 2023Unsupervised Speech Recognition with N-Skipgram and Positional Unigram Matchingcited by 5
  8. 2024Interface Design for Self-Supervised Speech Modelscited by 5
  9. 2023A Theory of Unsupervised Speech Recognitioncited by 4
  10. 2022Bridging Speech and Textual Pre-trained Models with Unsupervised ASRcited by 3
  11. 2022Introducing Semantics into Speech Encoderscited by 3
  12. 2022Learning the joint distribution of two sequences using little or no paired datacited by 3
  13. 2023Textless Speech-to-Speech Translation With Limited Parallel Datacited by 3
  14. 2023Active Bird2Vec: Towards End-to-End Bird Sound Monitoring with Transformerscited by 3
  15. 2024REBORN: Reinforcement-Learned Boundary Segmentation with Iterative Training for Unsupervised ASRcited by 3
  16. 2024Improving Whisper's Recognition Performance for Under-Represented Language Kazakh Leveraging Unpaired Speech and Textcited by 3
  17. 2024Self-supervised Speech Models for Word-Level Stuttered Speech Detectioncited by 3
  18. 2022Bootstrapping meaning through listening: Unsupervised learning of spoken sentence embeddingscited by 2
  19. 2022Once-for-All Sequence Compression for Self-Supervised Speech Modelscited by 2
  20. 2025Towards Unsupervised Speech Recognition at the Syllable-Levelcited by 2
  21. 2023Towards Matching Phones and Speech Representationscited by 1
  22. 2024Towards Unsupervised Speech Recognition Without Pronunciation Modelscited by 1
  23. 2025How I Built ASR for Endangered Languages with a Spoken Dictionarycited by 1
  24. 2022Training Autoregressive Speech Recognition Models with Limited in-domain Supervisioncited by 0
  25. 2023BayesSpeech: A Bayesian Transformer Network for Automatic Speech Recognitioncited by 0
  26. 2023Unsupervised ASR via Cross-Lingual Pseudo-Labelingcited by 0
  27. 2023Research on an improved Conformer end-to-end Speech Recognition Model with R-Drop Structurecited by 0
  28. 2024Maximizing Data Efficiency for Cross-Lingual TTS Adaptation by Self-Supervised Representation Mixing and Embedding Initializationcited by 0
  29. 2025Unifying Model and Layer Fusion for Speech Foundation Modelscited by 0
  30. 2025LibriVAD: A Scalable Open Dataset with Deep Learning Benchmarks for Voice Activity Detectioncited by 0
  31. 2026Sequence-Level Unsupervised Training in Speech Recognition: A Theoretical Studycited by 0
  32. 2026Improving End-to-End Speech Recognition for Dysarthric Speech through In-Domain Data Augmentationcited by 0