mlsanityData to 14 Oct 2026

Dynamic Frequency-Adaptive Knowledge Distillation for Speech Enhancement

arXiv 2502.04711 ↗First posted 7 Feb 2025cs.SD eess.AScited by 2

On the map: Knowledge distillation → Speech and audio distillation

Abstract (arXiv)

Deep learning-based speech enhancement (SE) models have recently outperformed traditional techniques, yet their deployment on resource-constrained devices remains challenging due to high computational and memory demands. This paper introduces a novel dynamic frequency-adaptive knowledge distillation (DFKD) approach to effectively compress SE models. Our method dynamically assesses the model's output, distinguishing between high and low-frequency components, and adapts the learning objectives to meet the unique requirements of different frequency bands, capitalizing on the SE task's inherent characteristics. To evaluate the DFKD's efficacy, we conducted experiments on three state-of-the-art models: DCCRN, ConTasNet, and DPTNet. The results demonstrate that our method not only significantly enhances the performance of the compressed model (student model) but also surpasses other logit-based knowledge distillation methods specifically for SE tasks.

Key facts. Cited by 2 papers in the map (data through 14 Oct 2026). Placed in Speech and audio distillation, one of 12 directions in Knowledge distillation. Builds on 22 of its 31 references that are in the map. First posted on arXiv 7 Feb 2025.

Builds on

22 of 31 references.

Closest ancestors · 2

Same direction, newest first

  1. 2023ABC-KD: Attention-Based-Compression Knowledge Distillation for Deep Learning-Based Noise Suppression26 May
  2. 2022Fast Real-time Personalized Speech Enhancement: End-to-End Enhancement Network (E3Net) and Knowledge Distillation2 Apr

Other references · 20

  1. 2015Distilling the Knowledge in a Neural Networkcited by 10,324
  2. 2015Learning both Weights and Connections for Efficient Neural Networkscited by 2,031
  3. 2016Paying More Attention to Attention: Improving the Performance of Convolutional Neural Networks via Attention Transfercited by 1,077
  4. 2017Learning Efficient Convolutional Networks through Network Slimmingcited by 676
  5. 2018Conv-TasNet: Surpassing Ideal Time-Frequency Magnitude Masking for Speech Separationcited by 406
  6. 2019A Comprehensive Overhaul of Feature Distillationcited by 348
  7. 2019Correlation Congruence for Knowledge Distillationcited by 285
  8. 2018Paraphrasing Complex Network: Network Compression via Factor Transfercited by 260
  9. 2018Knowledge Transfer via Distillation of Activation Boundaries Formed by Hidden Neuronscited by 238
  10. 2017Learning to Prune Deep Neural Networks via Layer-wise Optimal Brain Surgeoncited by 221
  11. 2020Cross-Layer Distillation with Semantic Calibrationcited by 143
  12. 2018Scalable Methods for 8-bit Training of Neural Networkscited by 142
8 more
  1. 2020DCCRN: Deep Complex Convolution Recurrent Network for Phase-Aware Speech Enhancementcited by 141
  2. 2020The INTERSPEECH 2020 Deep Noise Suppression Challenge: Datasets, Subjective Testing Framework, and Challenge Resultscited by 133
  3. 2020Dual-Path Transformer Network: Direct Context-Aware Modeling for End-to-End Monaural Speech Separationcited by 68
  4. 2019Demucs: Deep Extractor for Music Sources with extra unlabeled data remixedcited by 54
  5. 2018Learning Student Networks via Feature Embeddingcited by 45
  6. 2020Differentiable Feature Aggregation Search for Knowledge Distillationcited by 23
  7. 2019Adaptive Regularization of Labelscited by 16
  8. 2020Sub-Band Knowledge Distillation Framework for Speech Enhancementcited by 7

Built on by

Cited by year

Citing papers per year: 2025: 2.

2 papers.

  1. 2025Leveraging Local and Global Knowledge Integration with Time-Frequency Calibrated Distillation for Speech Enhancementcited by 0
  2. 2025DISPATCH: Distilling Selective Patches for Speech Enhancementcited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2025Comparison of Knowledge Distillation Methods for Low-complexity Multi-microphone Speech Enhancement using the FT-JNF Architecture
  2. 2024Distil-DCCRN: A Small-footprint DCCRN Leveraging Feature-based Knowledge Distillation in Speech Enhancement
  3. 2023Two-Step Knowledge Distillation for Tiny Speech Enhancement
  4. 2026RT-SEMamba: Real-Time Speech Enhancement Mamba via Progressive Knowledge Distillation
  5. 2026Knowledge Distillation for Efficient Acoustic Echo Control
  6. 2026ASKD-Whisper: Adaptive Self-knowledge Distillation for Efficient and Low-Latency Automatic Speech Recognition
  7. 2024Modulating State Space Model with SlowFast Framework for Compute-Efficient Ultra Low-Latency Speech Enhancement
  8. 2021Knowledge distillation from language model to acoustic model: a hierarchical multi-task learning approach