mlsanityData to 7 Oct 2026

An Unsupervised Autoregressive Model for Speech Representation Learning

arXiv 1904.03240 ↗First posted 5 Apr 2019cs.CL cs.LG cs.SDcited by 185

Not on the map. Why?

Abstract (arXiv)

This paper proposes a novel unsupervised autoregressive neural model for learning generic speech representations. In contrast to other speech representation learning methods that aim to remove noise or speaker variabilities, ours is designed to preserve information for a wide range of downstream tasks. In addition, the proposed model does not require any phonetic or word boundary labels, allowing the model to benefit from large quantities of unlabeled data. Speech representations learned by our model significantly improve performance on both phone classification and speaker verification over the surface features and other supervised and unsupervised approaches. Further analysis shows that different levels of speech information are captured by our model at different layers. In particular, the lower layers tend to be more discriminative for speakers, while the upper layers provide more phonetic content.

Key facts. Cited by 185 papers in the map (data through 7 Oct 2026). First posted on arXiv 5 Apr 2019.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2021: 59; 2022: 61; 2023: 27; 2024: 22; 2025: 10; 2026: 6.

185 papers.

Sort:

By area:

  1. 2021HuBERT: Self-Supervised Speech Representation Learning by Masked Prediction of Hidden Unitscited by 2,301
  2. 2021WavLM: Large-Scale Self-Supervised Pre-Training for Full Stack Speech Processingcited by 1,609
  3. 2022data2vec: A General Framework for Self-supervised Learning in Speech, Vision and Languagecited by 718
  4. 2021SUPERB: Speech processing Universal PERformance Benchmarkcited by 652
  5. 2021W2v-BERT: Combining Contrastive Learning and Masked Language Modeling for Self-Supervised Speech Pre-Trainingcited by 340
  6. 2021Generative Spoken Language Modeling from Raw Audiocited by 336
  7. 2022Learning Audio-Visual Speech Representation by Masked Multimodal Cluster Predictioncited by 277
  8. 2022Self-Supervised Speech Representation Learning: A Reviewcited by 258
  9. 2023Google USM: Scaling Automatic Speech Recognition Beyond 100 Languagescited by 228
  10. 2021SSAST: Self-Supervised Audio Spectrogram Transformercited by 166
  11. 2021Robust wav2vec 2.0: Analyzing Domain Shift in Self-Supervised Pre-Trainingcited by 148
  12. 2021Unsupervised Speech Recognitioncited by 147
38 more
  1. 2021Self-Supervised Representation Learning: Introduction, Advances and Challengescited by 147
  2. 2021Recent Advances in End-to-End Automatic Speech Recognitioncited by 138
  3. 2021DistilHuBERT: Speech Representation Learning by Layer-wise Distillation of Hidden-unit BERTcited by 129
  4. 2021Text-Free Prosody-Aware Generative Spoken Language Modelingcited by 119
  5. 2021BigSSL: Exploring the Frontier of Large-Scale Semi-Supervised Learning for Automatic Speech Recognitioncited by 88
  6. 2021A Fine-tuned Wav2vec 2.0/HuBERT Benchmark For Speech Emotion Recognition, Speaker Verification and Spoken Language Understandingcited by 82
  7. 2021PSLA: Improving Audio Tagging with Pretraining, Sampling, Labeling, and Aggregationcited by 81
  8. 2021Self-supervised representation learning from 12-lead ECG datacited by 80
  9. 2021Simple and Effective Zero-shot Cross-lingual Phoneme Recognitioncited by 79
  10. 2021UniSpeech: Unified Speech Representation Learning with Labeled and Unlabeled Datacited by 76
  11. 2022SUPERB-SG: Enhanced Speech processing Universal PERformance Benchmark for Semantic and Generative Capabilitiescited by 76
  12. 2022ContentVec: An Improved Self-Supervised Speech Representation by Disentangling Speakerscited by 73
  13. 2022A Comprehensive Survey on Deep Clustering: Taxonomy, Challenges, and Future Directionscited by 73
  14. 2021SLUE: New Benchmark Tasks for Spoken Language Understanding Evaluation on Natural Speechcited by 70
  15. 2022Audio Self-supervised Learning: A Surveycited by 65
  16. 2022Assessing the State of Self-Supervised Human Activity Recognition using Wearablescited by 63
  17. 2021PARP: Prune, Adjust and Re-Prune for Self-Supervised Speech Recognitioncited by 50
  18. 2022SpeechLM: Enhanced Speech Pre-Training with Unpaired Textual Datacited by 49
  19. 2021LeBenchmark: A Reproducible Framework for Assessing Self-Supervised Representation Learning from Speechcited by 46
  20. 2021UniSpeech-SAT: Universal Speech Representation Learning with Speaker Aware Pre-Trainingcited by 43
  21. 2022On the Utility of Self-supervised Models for Prosody-related Taskscited by 42
  22. 2021Self-supervised Contrastive Cross-Modality Representation Learning for Spoken Question Answeringcited by 39
  23. 2021An Exploration of Self-Supervised Pretrained Representations for End-to-End Speech Recognitioncited by 39
  24. 2024A Large-Scale Evaluation of Speech Foundation Modelscited by 38
  25. 2022Self-supervised models of audio effectively explain human cortical responses to speechcited by 37
  26. 2024SSAMBA: Self-Supervised Audio Representation Learning with Mamba State Space Modelcited by 36
  27. 2022TranSpeech: Speech-to-Speech Translation With Bilateral Perturbationcited by 34
  28. 2022VATLM: Visual-Audio-Text Pre-Training with Unified Masked Prediction for Speech Representation Learningcited by 34
  29. 2023DinoSR: Self-Distillation and Online Clustering for Self-supervised Speech Representation Learningcited by 34
  30. 2021Universal Paralinguistic Speech Representations Using Self-Supervised Conformerscited by 32
  31. 2022Efficient Adapter Transfer of Self-Supervised Speech Models for Automatic Speech Recognitioncited by 31
  32. 2022Exploration of A Self-Supervised Speech Model: A Study on Emotional Corporacited by 31
  33. 2022SpeechCLIP: Integrating Speech with Pre-Trained Vision and Language Modelcited by 29
  34. 2022Deep versus Wide: An Analysis of Student Architectures for Task-Agnostic Knowledge Distillation of Self-Supervised Speech Modelscited by 28
  35. 2022SUPERB @ SLT 2022: Challenge on Generalization and Efficiency of Self-Supervised Speech Representation Learningcited by 26
  36. 2022Self-Supervised Representation Learning for Speech Using Visual Grounding and Masked Language Modelingcited by 25
  37. 2022Word Segmentation on Discovered Phone Units with Dynamic Programming and Self-Supervised Scoringcited by 24
  38. 2021Utilizing Self-supervised Representations for MOS Predictioncited by 20

50 most cited of 185 All 185 (JSON)