# Towards Data-free and Training-free Compression for Speech Foundation Models Using Parameter Clustering

arXiv: https://arxiv.org/abs/2606.11836 · HTML: https://mlsanity.com/paper/2606.11836/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 26 of its 32 references that are in the map.
- First posted on arXiv 10 Jun 2026.

## Builds on (most cited first, 26 of 26)

- [Robust Speech Recognition via Large-Scale Weak Supervision](https://mlsanity.com/paper/2212.04356/) (2022)
- [wav2vec 2.0: A Framework for Self-Supervised Learning of Speech Representations](https://mlsanity.com/paper/2006.11477/) (2020)
- [HuBERT: Self-Supervised Speech Representation Learning by Masked Prediction of Hidden Units](https://mlsanity.com/paper/2106.07447/) (2021)
- [Learning both Weights and Connections for Efficient Neural Networks](https://mlsanity.com/paper/1506.02626/) (2015)
- [WavLM: Large-Scale Self-Supervised Pre-Training for Full Stack Speech Processing](https://mlsanity.com/paper/2110.13900/) (2021)
- [I-BERT: Integer-only BERT Quantization](https://mlsanity.com/paper/2101.01321/) (2021)
- [DistilHuBERT: Speech Representation Learning by Layer-wise Distillation of Hidden-unit BERT](https://mlsanity.com/paper/2110.01900/) (2021)
- [LightHuBERT: Lightweight and Configurable Speech Representation Learning with Once-for-All Hidden-Unit BERT](https://mlsanity.com/paper/2203.15610/) (2022)
- [DPHuBERT: Joint Distillation and Pruning of Self-Supervised Speech Models](https://mlsanity.com/paper/2305.17651/) (2023)
- [Deep versus Wide: An Analysis of Student Architectures for Task-Agnostic Knowledge Distillation of Self-Supervised Speech Models](https://mlsanity.com/paper/2207.06867/) (2022)
- [Structured Pruning of Self-Supervised Pre-trained Models for Speech Recognition and Understanding](https://mlsanity.com/paper/2302.14132/) (2023)
- [FitHuBERT: Going Thinner and Deeper for Knowledge Distillation of Speech Self-Supervised Learning](https://mlsanity.com/paper/2207.00555/) (2022)
- [4-bit Conformer with Native Quantization Aware Training for Speech Recognition](https://mlsanity.com/paper/2203.15952/) (2022)
- [Task-Agnostic Structured Pruning of Speech Representation Models](https://mlsanity.com/paper/2306.01385/) (2023)
- [Efficient conformer-based speech recognition with linear attention](https://mlsanity.com/paper/2104.06865/) (2021)
- [Layer Pruning on Demand with Intermediate CTC](https://mlsanity.com/paper/2106.09216/) (2021)
- [PADA: Pruning Assisted Domain Adaptation for Self-Supervised Speech Representations](https://mlsanity.com/paper/2203.16965/) (2022)
- [Accurate and Structured Pruning for Efficient Automatic Speech Recognition](https://mlsanity.com/paper/2305.19549/) (2023)
- [2-bit Conformer quantization for automatic speech recognition](https://mlsanity.com/paper/2305.16619/) (2023)
- [A Model for Every User and Budget: Label-Free and Personalized Mixed-Precision Quantization](https://mlsanity.com/paper/2307.12659/) (2023)
- [DistillW2V2: A Small and Streaming Wav2vec 2.0 Based ASR Model](https://mlsanity.com/paper/2303.09278/) (2023)
- [Effective and Efficient Mixed Precision Quantization of Speech Foundation Models](https://mlsanity.com/paper/2501.03643/) (2025)
- [One-pass Multiple Conformer and Foundation Speech Systems Compression and Quantization Using An All-in-one Neural Model](https://mlsanity.com/paper/2406.10160/) (2024)
- [Towards One-bit ASR: Extremely Low-bit Conformer Quantization Using Co-training and Stochastic Precision](https://mlsanity.com/paper/2505.21245/) (2025)
- [Unfolding A Few Structures for The Many: Memory-Efficient Compression of Conformer and Speech Foundation Models](https://mlsanity.com/paper/2505.21237/) (2025)
- [Effective and Efficient One-pass Compression of Speech Foundation Models Using Sparsity-aware Self-pinching Gates](https://mlsanity.com/paper/2505.22608/) (2025)

## Abstract (arXiv)

This paper presents a novel data-free and training-free compression approach for speech foundation models using channelwise clustering via k-means. More fine-grained, mixed sparsity pruning by layer-level varying number of parameter clusters is also explored. Experiments conducted on the LibriSpeech dataset suggest that when operating with pruning sparsity of 50% on HuBERT-large, consistent WER reductions of 27.73%/18.61% absolute (34.37%/21.91% relative) over the magnitude-based pruning were obtained on the test-clean and test-other subsets before fine-tuning and 0.19%/0.79% absolute (3.36%/4.62% relative) after fine-tuning with only 3 epochs. Similar WER reductions of 2.86%/5.02% absolute (59.21%/55.29% relative) were observed against magnitudebased pruning on Whisper-large-v3 at 10% sparsity, all with no significant WER increase relative to the uncompressed baseline.