mlsanityData to 7 Oct 2026

A Theory of Unsupervised Speech Recognition

arXiv 2306.07926 ↗First posted 9 Jun 2023eess.AS cs.CL cs.LGcited by 4

Not on the map. Why?

Abstract (arXiv)

Unsupervised speech recognition (ASR-U) is the problem of learning automatic speech recognition (ASR) systems from unpaired speech-only and text-only corpora. While various algorithms exist to solve this problem, a theoretical framework is missing from studying their properties and addressing such issues as sensitivity to hyperparameters and training instability. In this paper, we proposed a general theoretical framework to study the properties of ASR-U systems based on random matrix theory and the theory of neural tangent kernels. Such a framework allows us to prove various learnability conditions and sample complexity bounds of ASR-U. Extensive ASR-U experiments on synthetic languages with three classes of transition graphs provide strong empirical evidence for our theory (code available at cactuswiththoughts/UnsupASRTheory.git).

Key facts. Cited by 4 papers in the map (data through 7 Oct 2026). Builds on 20 of its 41 references that are in the map. First posted on arXiv 9 Jun 2023.

Builds on

20 of 41 references.

References · 20

  1. 2020wav2vec 2.0: A Framework for Self-Supervised Learning of Speech Representationscited by 3,853
  2. 2018Neural Tangent Kernel: Convergence and Generalization in Neural Networkscited by 2,597
  3. 2018Which Training Methods for GANs do actually Converge?cited by 672
  4. 2018Gradient Descent Provably Optimizes Over-parameterized Neural Networkscited by 655
  5. 2017Wasserstein GANcited by 635
  6. 2017MMD GAN: Towards Deeper Understanding of Moment Matching Networkcited by 251
  7. 2017Generalization and Equilibrium in Generative Adversarial Nets (GANs)cited by 165
  8. 2021Unsupervised Speech Recognitioncited by 147
  9. 2017On the Properties of the Softmax Function with Application in Game Theory and Reinforcement Learningcited by 136
  10. 2017The Numerics of GANscited by 104
  11. 2022Towards End-to-end Unsupervised Speech Recognitioncited by 44
  12. 2018Approximability of Discriminators Implies Diversity in GANscited by 28
8 more
  1. 2018Completely Unsupervised Phoneme Recognition by Adversarially Learning Mapping Relationships from Audio Embeddingscited by 22
  2. 2020On the Convergence of Gradient Descent in GANs: MMD GAN As a Gradient Flowcited by 21
  3. 2021A Neural Tangent Kernel Perspective of GANscited by 21
  4. 2018Unsupervised Speech Recognition via Segmental Empirical Output Distribution Matchingcited by 20
  5. 2021Understanding Overparameterization in Generative Adversarial Networkscited by 14
  6. 2022Unsupervised Text-to-Speech Synthesis by Unsupervised Automatic Speech Recognitioncited by 14
  7. 2021Analyzing the Robustness of Unsupervised Speech Recognitioncited by 11
  8. 2021Deciphering Speech: a Zero-Resource Approach to Cross-Lingual Transfer in ASRcited by 11

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2023: 1; 2024: 0; 2025: 1; 2026: 2.

4 papers.

  1. 2023Unsupervised Speech Recognition with N-Skipgram and Positional Unigram Matchingcited by 5
  2. 2025Towards Unsupervised Speech Recognition at the Syllable-Levelcited by 2
  3. 2026Using the Mimi codec for metalinguistic representationscited by 1
  4. 2026Sequence-Level Unsupervised Training in Speech Recognition: A Theoretical Studycited by 0