mlsanityData to 7 Oct 2026

Unsupervised Automatic Speech Recognition: A Review

arXiv 2106.04897 ↗First posted 9 Jun 2021cs.CL cs.SD eess.AScited by 5

Not on the map. Why?

Abstract (arXiv)

Automatic Speech Recognition (ASR) systems can be trained to achieve remarkable performance given large amounts of manually transcribed speech, but large labeled data sets can be difficult or expensive to acquire for all languages of interest. In this paper, we review the research literature to identify models and ideas that could lead to fully unsupervised ASR, including unsupervised segmentation of the speech signal, unsupervised mapping from speech segments to text, and semi-supervised models with nominal amounts of labeled examples. The objective of the study is to identify the limitations of what can be learned from speech data alone and to understand the minimum requirements for speech recognition. Identifying these limitations would help optimize the resources and efforts in ASR development for low-resource languages.

Key facts. Cited by 5 papers in the map (data through 7 Oct 2026). Builds on 52 of its 147 references that are in the map. First posted on arXiv 9 Jun 2021.

Builds on

52 of 147 references.

References · 52

  1. 2017Attention Is All You Needcited by 59,932
  2. 2015Deep Residual Learning for Image Recognitioncited by 51,809
  3. 2018Representation Learning with Contrastive Predictive Codingcited by 7,662
  4. 2013Efficient Estimation of Word Representations in Vector Spacecited by 4,346
  5. 2020wav2vec 2.0: A Framework for Self-Supervised Learning of Speech Representationscited by 3,853
  6. 2020Conformer: Convolution-augmented Transformer for Speech Recognitioncited by 1,838
  7. 2020A Survey on Contrastive Self-supervised Learningcited by 553
  8. 2017Word Translation Without Parallel Datacited by 480
  9. 2019Libri-Light: A Benchmark for ASR with Limited or No Supervisioncited by 462
  10. 2015Deep Speech 2: End-to-End Speech Recognition in English and Mandarincited by 449
  11. 2017State-of-the-art Speech Recognition With Sequence-to-Sequence Modelscited by 196
  12. 2019An Unsupervised Autoregressive Model for Speech Representation Learningcited by 185
38 more
  1. 2021Unsupervised Speech Recognitioncited by 147
  2. 2018A robust self-learning method for fully unsupervised cross-lingual mappings of word embeddingscited by 145
  3. 2019End-to-end ASR: from Supervised to Semi-Supervised Learning with Modern Architecturescited by 120
  4. 2016Hierarchical Multiscale Recurrent Neural Networkscited by 102
  5. 2020Unsupervised pretraining transfers well across languagescited by 100
  6. 2019Effectiveness of self-supervised pre-training for speech recognitioncited by 72
  7. 2017Speech Recognition Challenge in the Wild: Arabic MGB-3cited by 60
  8. 2015Deep Multimodal Semantic Embeddings for Speech and Imagescited by 56
  9. 2020Vector-quantized neural networks for acoustic unit discovery in the ZeroSpeech 2020 challengecited by 55
  10. 2018Speech2Vec: A Sequence-to-Sequence Framework for Learning Word Embeddings from Speechcited by 52
  11. 2017The Zero Resource Speech Challenge 2017cited by 45
  12. 2016Wav2Letter: an End-to-End ConvNet-based Speech Recognition Systemcited by 43
  13. 2020Self-Supervised Contrastive Learning for Unsupervised Phoneme Segmentationcited by 43
  14. 2020The Zero Resource Speech Challenge 2020: Discovering discrete subword and word unitscited by 39
  15. 2019The Zero Resource Speech Challenge 2019: TTS without Tcited by 38
  16. 2017Representations of language in a model of visually grounded speech signalcited by 37
  17. 2016Audio Word2Vec: Unsupervised Learning of Audio Segment Representations using Sequence-to-sequence Autoencodercited by 34
  18. 2018Unsupervised Cross-Modal Alignment of Speech and Text Embedding Spacescited by 34
  19. 2017An embedded segmental K-means model for unsupervised segmentation and clustering of speechcited by 31
  20. 2018Truly unsupervised acoustic word embeddings using weak top-down constraints in encoder-decoder modelscited by 29
  21. 2016A segmental framework for fully-unsupervised large-vocabulary speech recognitioncited by 27
  22. 2019Sequence-to-Sequence Speech Recognition with Time-Depth Separable Convolutionscited by 27
  23. 2020Towards unsupervised phone and word segmentation using self-supervised vector-quantized neural networkscited by 24
  24. 2018Completely Unsupervised Phoneme Recognition by Adversarially Learning Mapping Relationships from Audio Embeddingscited by 22
  25. 2018Unsupervised Speech Recognition via Segmental Empirical Output Distribution Matchingcited by 20
  26. 2017End-to-End ASR-free Keyword Search from Speechcited by 19
  27. 2018Segmental Audio Word2Vec: Representing Utterances as Sequences of Vectors with Applications in Spoken Term Detectioncited by 17
  28. 2020Unsupervised Discovery of Recurring Speech Patterns Using Probabilistic Adaptive Metricscited by 16
  29. 2019Towards Visually Grounded Sub-Word Speech Unit Discoverycited by 15
  30. 2019A Fully Differentiable Beam Search Decodercited by 14
  31. 2020A Correspondence Variational Autoencoder for Unsupervised Acoustic Word Embeddingscited by 12
  32. 2018Phonetic-and-Semantic Embedding of Spoken Words with Applications in Spoken Content Retrievalcited by 11
  33. 2020A comparison of self-supervised speech representations as input features for unsupervised acoustic word embeddingscited by 10
  34. 2016Blind phoneme segmentation with temporal prediction errorscited by 9
  35. 2017Gate Activation Signal Analysis for Gated Recurrent Neural Networks and Its Correlation with Phoneme Boundariescited by 9
  36. 2019On the Use/Misuse of the Term 'Phoneme'cited by 9
  37. 2019Completely Unsupervised Speech Recognition By A Generative Adversarial Network Harmonized With Iteratively Refined Hidden Markov Modelscited by 8
  38. 2017Unsupervised Word Mapping Using Structural Similarities in Monolingual Embeddingscited by 7

50 most cited of 52 All 52 (JSON)

Built on by

Cited by year

Citing papers per year: 2023: 4; 2024: 1.

5 papers.

  1. 2024Automatic Speech Recognition using Advanced Deep Learning Approaches: A surveycited by 50
  2. 2023NoRefER: a Referenceless Quality Metric for Automatic Speech Recognition via Semi-Supervised Language Model Fine-Tuning with Contrastive Learningcited by 4
  3. 2023Supervised Acoustic Embeddings And Their Transferability Across Languagescited by 1
  4. 2023Unsupervised ASR via Cross-Lingual Pseudo-Labelingcited by 0
  5. 2023Spoken Word2Vec: Learning Skipgram Embeddings from Speechcited by 0