HuBERT-VIC: Improving Noise-Robust Automatic Speech Recognition of Speech Foundation Model via Variance-Invariance-Covariance Regularization
Not on the map. Why?
Abstract (arXiv)
Noise robustness in speech foundation models (SFMs) has been a critical challenge, as most models are primarily trained on clean data and experience performance degradation when the models are exposed to noisy speech. To address this issue, we propose HuBERT-VIC, a noise-robust SFM with variance, in-variance, and covariance regularization (VICReg) objectives. These objectives adjust the statistics of noisy speech representations, enabling the model to capture diverse acoustic characteristics and improving the generalization ability across different types of noise. When applied to HuBERT, our model shows relative performance improvements of 23.3% on LibriSpeech test-clean and 13.2% on test-other, compared to the baseline model pre-trained on noisy speech.
Key facts. Cited by 2 papers in the map (data through 14 Oct 2026). Builds on 24 of its 28 references that are in the map. First posted on arXiv 17 Aug 2025.
Builds on
24 of 28 references.
References · 24
- 2017Attention Is All You Needcited by 60,361
- 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,399
- 2020wav2vec 2.0: A Framework for Self-Supervised Learning of Speech Representationscited by 3,905
- 2021HuBERT: Self-Supervised Speech Representation Learning by Masked Prediction of Hidden Unitscited by 2,342
- 2021WavLM: Large-Scale Self-Supervised Pre-Training for Full Stack Speech Processingcited by 1,657
- 2019fairseq: A Fast, Extensible Toolkit for Sequence Modelingcited by 1,488
- 2021VICReg: Variance-Invariance-Covariance Regularization for Self-Supervised Learningcited by 1,057
- 2022data2vec: A General Framework for Self-supervised Learning in Speech, Vision and Languagecited by 727
- 2015MUSAN: A Music, Speech, and Noise Corpuscited by 425
- 2021W2v-BERT: Combining Contrastive Learning and Masked Language Modeling for Self-Supervised Speech Pre-Trainingcited by 347
- 2019Mockingjay: Unsupervised Speech Representation Learning with Deep Bidirectional Transformer Encoderscited by 198
- 2020TERA: Self-Supervised Learning of Transformer Encoder Representation for Speechcited by 188
12 more
- 2020DeCoAR 2.0: Deep Contextualized Acoustic Representations with Vector Quantizationcited by 77
- 2022Robust Self-Supervised Audio-Visual Speech Recognitioncited by 75
- 2021Wav2vec-C: A Self-supervised Model for Speech Representation Learningcited by 25
- 2022A Noise-Robust Self-supervised Pre-training Model Based Speech Representation Learning for Automatic Speech Recognitioncited by 22
- 2022Robust Data2vec: Noise-robust Speech Representation Learning for ASR by Combining Regression and Improved Contrastive Learningcited by 22
- 2021Wav2vec-Switch: Contrastive Learning from Original-noisy Speech Pairs for Robust Speech Recognitioncited by 16
- 2021Improving Noise Robustness of Contrastive Speech Representation Learning with Speech Reconstructioncited by 13
- 2022Joint Training of Speech Enhancement and Self-supervised Model for Noise-robust ASRcited by 9
- 2023deHuBERT: Disentangling Noise in a Self-supervised Model for Robust Speech Recognitioncited by 7
- 2024Learning Video Temporal Dynamics with Cross-Modal Attention for Robust Audio-Visual Speech Recognitioncited by 7
- 2022I2CR: Improving Noise Robustness on Keyword Spotting Using Inter-Intra Contrastive Regularizationcited by 4
- 2023STaR: Distilling Speech Temporal Relation for Lightweight Speech Self-Supervised Learning Modelscited by 3
Built on by
Citing papers per year: 2025: 1; 2026: 1.
2 papers.