mlsanityData to 13 Oct 2026

Fast-HuBERT: An Efficient Training Framework for Self-Supervised Speech Representation Learning

arXiv 2309.13860 ↗First posted 25 Sep 2023cs.CL cs.AI cs.LGcited by 9

Not on the map. Why?

Abstract (arXiv)

Recent years have witnessed significant advancements in self-supervised learning (SSL) methods for speech-processing tasks. Various speech-based SSL models have been developed and present promising performance on a range of downstream tasks including speech recognition. However, existing speech-based SSL models face a common dilemma in terms of computational cost, which might hinder their potential application and in-depth academic research. To address this issue, we first analyze the computational cost of different modules during HuBERT pre-training and then introduce a stack of efficiency optimizations, which is named Fast-HuBERT in this paper. The proposed Fast-HuBERT can be trained in 1.1 days with 8 V100 GPUs on the Librispeech 960h benchmark, without performance degradation, resulting in a 5.2x speedup, compared to the original implementation. Moreover, we explore two well-studied techniques in the Fast-HuBERT and demonstrate consistent improvements as reported in previous work.

Key facts. Cited by 9 papers in the map (data through 13 Oct 2026). Builds on 21 of its 28 references that are in the map. First posted on arXiv 25 Sep 2023.

Builds on

21 of 28 references.

References · 21

  1. 2017Attention Is All You Needcited by 60,106
  2. 2020wav2vec 2.0: A Framework for Self-Supervised Learning of Speech Representationscited by 3,870
  3. 2021HuBERT: Self-Supervised Speech Representation Learning by Masked Prediction of Hidden Unitscited by 2,313
  4. 2021WavLM: Large-Scale Self-Supervised Pre-Training for Full Stack Speech Processingcited by 1,625
  5. 2019fairseq: A Fast, Extensible Toolkit for Sequence Modelingcited by 1,487
  6. 2019SpecAugment: A Simple Data Augmentation Method for Automatic Speech Recognitioncited by 1,359
  7. 2022data2vec: A General Framework for Self-supervised Learning in Speech, Vision and Languagecited by 721
  8. 2021SUPERB: Speech processing Universal PERformance Benchmarkcited by 657
  9. 2021Unsupervised Speech Recognitioncited by 147
  10. 2021Recent Advances in End-to-End Automatic Speech Recognitioncited by 138
  11. 2022Efficient Self-supervised Learning with Contextualized Target Representations for Vision, Speech and Languagecited by 108
  12. 2021Large-scale Self-Supervised Speech Representation Learning for Automatic Speaker Verificationcited by 72
9 more
  1. 2022Towards End-to-end Unsupervised Speech Recognitioncited by 45
  2. 2021Self-Supervised Learning for speech recognition with Intermediate layer supervisioncited by 15
  3. 2022Speech Pre-training with Acoustic Piececited by 14
  4. 2022Supervision-Guided Codebooks for Masked Prediction in Speech Pre-trainingcited by 14
  5. 2023Leveraging Speech PTM, Text LLM, and Emotional TTS for Speech Emotion Recognitioncited by 12
  6. 2022MT4SSL: Boosting Self-Supervised Speech Representation Learning by Integrating Multiple Targetscited by 8
  7. 2022CoBERT: Self-Supervised Speech Representation Learning Through Code Representation Learningcited by 7
  8. 2023Pushing the Limits of Unsupervised Unit Discovery for SSL Speech Representationcited by 7
  9. 2022CTCBERT: Advancing Hidden-unit BERT with CTC Objectivescited by 5

Built on by

Cited by year (2026 to 13 Oct)

Citing papers per year: 2024: 4; 2025: 3; 2026: 2.

9 papers.

  1. 2024An Embarrassingly Simple Approach for LLM with Strong ASR Capacitycited by 88
  2. 2024Towards Robust Speech Representation Learning for Thousands of Languagescited by 48
  3. 2024MuPT: A Generative Symbolic Music Pretrained Transformercited by 32
  4. 2025SpidR: Learning Fast and Stable Linguistic Units for Spoken Language Models Without Supervisioncited by 6
  5. 2024DAISY: Data Adaptive Self-Supervised Early Exit for Speech Representation Modelscited by 4
  6. 2025VietASR: Achieving Industry-level Vietnamese ASR with 50-hour labeled data and Large-Scale Speech Pretrainingcited by 3
  7. 2025Speech-FT: Merging Pre-trained And Fine-Tuned Speech Representation Models For Cross-Task Generalizationcited by 2
  8. 2026MSR-HuBERT: Self-supervised Pre-training for Adaptation to Multiple Sampling Ratescited by 0
  9. 2026Dissecting Sensitivity to Training Language in Self-Supervised Speech Learning Using Neural Audio Codec Tokenscited by 0