mlsanityData to 7 Oct 2026

Autoregressive Co-Training for Learning Discrete Speech Representations

arXiv 2203.15840 ↗First posted 29 Mar 2022cs.CLcited by 8

Not on the map. Why?

Abstract (arXiv)

While several self-supervised approaches for learning discrete speech representation have been proposed, it is unclear how these seemingly similar approaches relate to each other. In this paper, we consider a generative model with discrete latent variables that learns a discrete representation for speech. The objective of learning the generative model is formulated as information-theoretic co-training. Besides the wide generality, the objective can be optimized with several approaches, subsuming HuBERT-like training and vector quantization for learning discrete representation. Empirically, we find that the proposed approach learns discrete representation that is highly correlated with phonetic units, more correlated than HuBERT-like training and vector quantization.

Key facts. Cited by 8 papers in the map (data through 7 Oct 2026). Builds on 23 of its 25 references that are in the map. First posted on arXiv 29 Mar 2022.

Builds on

23 of 25 references.

References · 23

  1. 2017Neural Discrete Representation Learningcited by 4,650
  2. 2020wav2vec 2.0: A Framework for Self-Supervised Learning of Speech Representationscited by 3,853
  3. 2016Categorical Reparameterization with Gumbel-Softmaxcited by 2,708
  4. 2021HuBERT: Self-Supervised Speech Representation Learning by Masked Prediction of Hidden Unitscited by 2,301
  5. 2016The Concrete Distribution: A Continuous Relaxation of Discrete Random Variablescited by 1,135
  6. 2021W2v-BERT: Combining Contrastive Learning and Masked Language Modeling for Self-Supervised Speech Pre-Trainingcited by 340
  7. 2019vq-wav2vec: Self-Supervised Learning of Discrete Speech Representationscited by 335
  8. 2019An Unsupervised Autoregressive Model for Speech Representation Learningcited by 185
  9. 2014Neural Variational Inference and Learning in Belief Networkscited by 116
  10. 2019Unsupervised speech representation learning using WaveNet autoencoderscited by 90
  11. 2019Generative Pre-Training for Speech with Autoregressive Predictive Codingcited by 82
  12. 2020DeCoAR 2.0: Deep Contextualized Acoustic Representations with Vector Quantizationcited by 77
11 more
  1. 2020Vector-Quantized Autoregressive Predictive Codingcited by 69
  2. 2020Vector-quantized neural networks for acoustic unit discovery in the ZeroSpeech 2020 challengecited by 55
  3. 2020Non-Autoregressive Predictive Coding for Learning Speech Representations from Local Dependenciescited by 47
  4. 2019Learning Hierarchical Discrete Linguistic Units from Visually-Grounded Speechcited by 44
  5. 2021Segmental Contrastive Predictive Coding for Unsupervised Word Segmentationcited by 32
  6. 2021Aligned Contrastive Predictive Codingcited by 15
  7. 2018Deep Encoder-Decoder Models for Unsupervised Learning of Controllable Speech Synthesiscited by 13
  8. 2020Discrete Latent Variable Representations for Low-Resource Text Classificationcited by 10
  9. 2020A Comparison of Discrete Latent Variable Models for Speech Representation Learningcited by 8
  10. 2018Information Theoretic Co-Trainingcited by 7
  11. 2014Building DNN Acoustic Models for Large Vocabulary Speech Recognitioncited by 6

Built on by

Cited by year

Citing papers per year: 2022: 2; 2023: 3; 2024: 1; 2025: 2.

8 papers.

  1. 2023DinoSR: Self-Distillation and Online Clustering for Self-supervised Speech Representation Learningcited by 34
  2. 2023Self-supervised Fine-tuning for Improved Content Representations by Speaker-invariant Clusteringcited by 21
  3. 2022MelHuBERT: A simplified HuBERT on Mel spectrogramscited by 15
  4. 2025SpidR: Learning Fast and Stable Linguistic Units for Spoken Language Models Without Supervisioncited by 6
  5. 2024Revisiting Self-supervised Learning of Speech Representation from a Mutual Information Perspectivecited by 5
  6. 2022Learning Dependencies of Discrete Speech Representations with Neural Hidden Markov Modelscited by 1
  7. 2023Efficiency-oriented approaches for self-supervised speech representation learningcited by 1
  8. 2025Learning Speech Representations with Variational Predictive Codingcited by 0