# deHuBERT: Disentangling Noise in a Self-supervised Model for Robust Speech Recognition

arXiv: https://arxiv.org/abs/2302.14597 · HTML: https://mlsanity.com/paper/2302.14597/

## Key facts

- Cited by 7 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 28 Feb 2023.

## Built on by (most cited first, 7 of 7)

- [Learning Video Temporal Dynamics with Cross-Modal Attention for Robust Audio-Visual Speech Recognition](https://mlsanity.com/paper/2407.03563/) (2024)
- [HuBERT-VIC: Improving Noise-Robust Automatic Speech Recognition of Speech Foundation Model via Variance-Invariance-Covariance Regularization](https://mlsanity.com/paper/2508.12292/) (2025)
- [ACA-Net: Towards Lightweight Speaker Verification using Asymmetric Cross Attention](https://mlsanity.com/paper/2305.12121/) (2023)
- [Speech Separation with Pretrained Frontend to Minimize Domain Mismatch](https://mlsanity.com/paper/2411.03085/) (2024)
- [Noise robust distillation of self-supervised speech models via correlation metrics](https://mlsanity.com/paper/2312.12153/) (2023)
- [Causal Self-supervised Pretrained Frontend with Predictive Code for Speech Separation](https://mlsanity.com/paper/2504.02302/) (2025)
- [Position-invariant Fine-tuning of Speech Enhancement Models with Self-supervised Speech Representations](https://mlsanity.com/paper/2601.21084/) (2026)

## Abstract (arXiv)

Existing self-supervised pre-trained speech models have offered an effective way to leverage massive unannotated corpora to build good automatic speech recognition (ASR). However, many current models are trained on a clean corpus from a single source, which tends to do poorly when noise is present during testing. Nonetheless, it is crucial to overcome the adverse influence of noise for real-world applications. In this work, we propose a novel training framework, called deHuBERT, for noise reduction encoding inspired by H. Barlow's redundancy-reduction principle. The new framework improves the HuBERT training algorithm by introducing auxiliary losses that drive the self- and cross-correlation matrix between pairwise noise-distorted embeddings towards identity matrix. This encourages the model to produce noise-agnostic speech representations. With this method, we report improved robustness in noisy environments, including unseen noises, without impairing the performance on the clean set.