# Exploration on HuBERT with Multiple Resolutions

arXiv: https://arxiv.org/abs/2306.01084 · HTML: https://mlsanity.com/paper/2306.01084/

## Key facts

- Cited by 7 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 1 Jun 2023.

## Built on by (most cited first, 7 of 7)

- [Findings of the 2023 ML-SUPERB Challenge: Pre-Training and Evaluation over More Languages and Beyond](https://mlsanity.com/paper/2310.05513/) (2023)
- [Joint Prediction and Denoising for Large-scale Multilingual Self-supervised Learning](https://mlsanity.com/paper/2309.15317/) (2023)
- [Evaluating Self-Supervised Speech Representations for Indigenous American Languages](https://mlsanity.com/paper/2310.03639/) (2023)
- [MS-HuBERT: Mitigating Pre-training and Inference Mismatch in Masked Language Modelling methods for learning Speech Representations](https://mlsanity.com/paper/2406.05661/) (2024)
- [JOOCI: a Framework for Learning Comprehensive Speech Representations](https://mlsanity.com/paper/2410.11086/) (2024)
- [Speech Representation Learning Revisited: The Necessity of Separate Learnable Parameters and Robust Data Augmentation](https://mlsanity.com/paper/2408.10557/) (2024)
- [Optimizing Speech Multi-View Feature Fusion through Conditional Computation](https://mlsanity.com/paper/2501.08057/) (2025)

## Abstract (arXiv)

Hidden-unit BERT (HuBERT) is a widely-used self-supervised learning (SSL) model in speech processing. However, we argue that its fixed 20ms resolution for hidden representations would not be optimal for various speech-processing tasks since their attributes (e.g., speaker characteristics and semantics) are based on different time scales. To address this limitation, we propose utilizing HuBERT representations at multiple resolutions for downstream tasks. We explore two approaches, namely the parallel and hierarchical approaches, for integrating HuBERT features with different resolutions. Through experiments, we demonstrate that HuBERT with multiple resolutions outperforms the original model. This highlights the potential of utilizing multiple resolutions in SSL models like HuBERT to capture diverse information from speech signals.