# Task-Aware Joint Pruning and Distillation for Efficient Audio Deepfake Detection

arXiv: https://arxiv.org/abs/2610.05264 · HTML: https://mlsanity.com/paper/2610.05264/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 28 of its 39 references that are in the map.
- First posted on arXiv 4 Oct 2026.

## Builds on (most cited first, 28 of 28)

- [wav2vec 2.0: A Framework for Self-Supervised Learning of Speech Representations](https://mlsanity.com/paper/2006.11477/) (2020)
- [Similarity of Neural Network Representations Revisited](https://mlsanity.com/paper/1905.00414/) (2019)
- [WavLM: Large-Scale Self-Supervised Pre-Training for Full Stack Speech Processing](https://mlsanity.com/paper/2110.13900/) (2021)
- [XLS-R: Self-supervised Cross-lingual Speech Representation Learning at Scale](https://mlsanity.com/paper/2111.09296/) (2021)
- [Learning Sparse Neural Networks through L0 Regularization](https://mlsanity.com/paper/1712.01312/) (2017)
- [Movement Pruning: Adaptive Sparsity by Fine-Tuning](https://mlsanity.com/paper/2005.07683/) (2020)
- [ASVspoof 2019: Future Horizons in Spoofed and Fake Audio Detection](https://mlsanity.com/paper/1904.05441/) (2019)
- [What Do Compressed Deep Neural Networks Forget?](https://mlsanity.com/paper/1911.05248/) (2019)
- [Structured Pruning Learns Compact and Accurate Models](https://mlsanity.com/paper/2204.00408/) (2022)
- [ASVspoof 2021: accelerating progress in spoofed and deepfake speech detection](https://mlsanity.com/paper/2109.00537/) (2021)
- [DistilHuBERT: Speech Representation Learning by Layer-wise Distillation of Hidden-unit BERT](https://mlsanity.com/paper/2110.01900/) (2021)
- [Does Audio Deepfake Detection Generalize?](https://mlsanity.com/paper/2203.16263/) (2022)
- [Automatic speaker verification spoofing and deepfake detection using wav2vec 2.0 and data augmentation](https://mlsanity.com/paper/2202.12233/) (2022)
- [ASVspoof 5: Crowdsourced Speech Data, Deepfakes, and Adversarial Attacks at Scale](https://mlsanity.com/paper/2408.08739/) (2024)
- [DPHuBERT: Joint Distillation and Pruning of Self-Supervised Speech Models](https://mlsanity.com/paper/2305.17651/) (2023)
- [Structured Pruning of Self-Supervised Pre-trained Models for Speech Recognition and Understanding](https://mlsanity.com/paper/2302.14132/) (2023)
- [XLSR-Mamba: A Dual-Column Bidirectional State Space Model for Spoofing Attack Detection](https://mlsanity.com/paper/2411.10027/) (2024)
- [Speech DF Arena: A Leaderboard for Speech DeepFake Detection Models](https://mlsanity.com/paper/2509.02859/) (2025)
- [Task-Agnostic Structured Pruning of Speech Representation Models](https://mlsanity.com/paper/2306.01385/) (2023)
- [Learning From Yourself: A Self-Distillation Method for Fake Speech Detection](https://mlsanity.com/paper/2303.01211/) (2023)
- [One-Class Knowledge Distillation for Spoofing Speech Detection](https://mlsanity.com/paper/2309.08285/) (2023)
- [Comprehensive Layer-wise Analysis of SSL Models for Audio Deepfake Detection](https://mlsanity.com/paper/2502.03559/) (2025)
- [Representation Selective Self-distillation and wav2vec 2.0 Feature Exploration for Spoof-aware Speaker Verification](https://mlsanity.com/paper/2204.02639/) (2022)
- [Post-training for Deepfake Speech Detection](https://mlsanity.com/paper/2506.21090/) (2025)
- [Fine-tune Before Structured Pruning: Towards Compact and Accurate Self-Supervised Models for Speaker Diarization](https://mlsanity.com/paper/2505.24111/) (2025)
- [SKILL: Similarity-aware Knowledge distILLation for Speech Self-Supervised Learning](https://mlsanity.com/paper/2402.16830/) (2024)
- [Hybrid Pruning: In-Situ Compression of Self-Supervised Speech Models for Speaker Verification and Anti-Spoofing](https://mlsanity.com/paper/2508.16232/) (2025)
- [DistilXLSR: A Light Weight Cross-Lingual Speech Representation Model](https://mlsanity.com/paper/2306.01303/) (2023)

## Abstract (arXiv)

Advances in speech synthesis have made deepfake speeches increasingly convincing, posing growing threats to security. While self-supervised learning (SSL) based detectors achieve state-of-the-art performance, their computational demands (typically 300M+ parameters) prevent deployment on resource-constrained devices. Existing compression methods, designed mainly for content-centric tasks, struggle to maintain competitive performance when directly adapted to deepfake detection. We propose a Task-Aware Joint Pruning and Distillation framework that combines cross-domain knowledge distillation with movement-guided structured pruning to transfer forgery-discriminative knowledge and preserve critical structures under aggressive compression. Our framework reduces the model to 31.9M parameters with 6.3× FLOPs reduction, with an average performance drop of only 1.30% across multiple datasets compared to the uncompressed baseline, demonstrating strong potential for on-device deployment.