# Fine-tune Before Structured Pruning: Towards Compact and Accurate Self-Supervised Models for Speaker Diarization

arXiv: https://arxiv.org/abs/2505.24111 · HTML: https://mlsanity.com/paper/2505.24111/

## Key facts

- Cited by 10 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 30 May 2025.

## Built on by (most cited first, 10 of 10)

- [SpeakerLM: End-to-End Versatile Speaker Diarization and Recognition with Multimodal Large Language Models](https://mlsanity.com/paper/2508.06372/) (2025)
- [MOSS-TTS Technical Report](https://mlsanity.com/paper/2603.18090/) (2026)
- [Efficient and Robust Speaker Diarization via Structured Pruning of Self-Supervised Models](https://mlsanity.com/paper/2506.18623/) (2025)
- [MOSS-VoiceGenerator: Create Realistic Voices with Natural Language Descriptions](https://mlsanity.com/paper/2603.28086/) (2026)
- [Benchmarking Diarization Models](https://mlsanity.com/paper/2509.26177/) (2025)
- [SE-DiCoW: Self-Enrolled Diarization-Conditioned Whisper](https://mlsanity.com/paper/2601.19194/) (2026)
- [TCG CREST System Description for the DISPLACE-M Challenge](https://mlsanity.com/paper/2603.02030/) (2026)
- [Efficiency-Performance Trade-offs in Neural Speaker Diarization via Structured Pruning and Low-Bit Quantization](https://mlsanity.com/paper/2606.14030/) (2026)
- [Child-Centric Voice Anonymization in Single and Multi-Speaker Speech via Domain-Adapted SSL Models](https://mlsanity.com/paper/2606.29897/) (2026)
- [Task-Aware Joint Pruning and Distillation for Efficient Audio Deepfake Detection](https://mlsanity.com/paper/2610.05264/) (2026)

## Abstract (arXiv)

Self-supervised learning (SSL) models like WavLM can be effectively utilized when building speaker diarization systems but are often large and slow, limiting their use in resource constrained scenarios. Previous studies have explored compression techniques, but usually for the price of degraded performance at high pruning ratios. In this work, we propose to compress SSL models through structured pruning by introducing knowledge distillation. Different from the existing works, we emphasize the importance of fine-tuning SSL models before pruning. Experiments on far-field single-channel AMI, AISHELL-4, and AliMeeting datasets show that our method can remove redundant parameters of WavLM Base+ and WavLM Large by up to 80% without any performance degradation. After pruning, the inference speeds on a single GPU for the Base+ and Large models are 4.0 and 2.6 times faster, respectively. Our source code is publicly available.