# Hybrid Pruning: In-Situ Compression of Self-Supervised Speech Models for Speaker Verification and Anti-Spoofing

arXiv: https://arxiv.org/abs/2508.16232 · HTML: https://mlsanity.com/paper/2508.16232/

## Key facts

- Cited by 3 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 22 Aug 2025.

## Built on by (most cited first, 3 of 3)

- [RAT: Reference-Augmented Training for ASV Anti-Spoofing](https://mlsanity.com/paper/2606.10908/) (2026)
- [What Do Deepfake Speech Detectors Actually Hear?](https://mlsanity.com/paper/2606.10912/) (2026)
- [Task-Aware Joint Pruning and Distillation for Efficient Audio Deepfake Detection](https://mlsanity.com/paper/2610.05264/) (2026)

## Abstract (arXiv)

Although large-scale self-supervised learning (SSL) models like WavLM have achieved state-of-the-art performance in speech processing, their significant size impedes deployment on resource-constrained devices. While structured pruning is a key technique for model compression, existing methods typically separate it from task-specific fine-tuning. This multi-stage approach struggles to create optimal architectures tailored for diverse downstream tasks. In this work, we introduce a unified framework that integrates structured pruning into the downstream fine-tuning process. Our framework unifies these steps, jointly optimizing for task performance and model sparsity in a single stage. This allows the model to learn a compressed architecture specifically for the end task, eliminating the need for complex multi-stage pipelines and knowledge distillation. Our pruned models achieve up to a 70% parameter reduction with negligible performance degradation on large-scale datasets, achieving equal error rates of 0.7%, 0.8%, and 1.6% on Vox1-O, -E, and -H, respectively. Furthermore, our approach demonstrates improved generalization in low-resource scenarios, reducing overfitting and achieving a state-of-the-art 3.7% EER on ASVspoof5.