mlsanityData to 7 Oct 2026

Adaptive Slimming for Scalable and Efficient Speech Enhancement

arXiv 2507.04879 ↗First posted 7 Jul 2025eess.AS cs.LG cs.SDcited by 4

Not on the map. Why?

Abstract (arXiv)

Speech enhancement (SE) enables robust speech recognition, real-time communication, hearing aids, and other applications where speech quality is crucial. However, deploying such systems on resource-constrained devices involves choosing a static trade-off between performance and computational efficiency. In this paper, we introduce dynamic slimming to DEMUCS, a popular SE architecture, making it scalable and input-adaptive. Slimming lets the model operate at different utilization factors (UF), each corresponding to a different performance/efficiency trade-off, effectively mimicking multiple model sizes without the extra storage costs. In addition, a router subnet, trained end-to-end with the backbone, determines the optimal UF for the current input. Thus, the system saves resources by adaptively selecting smaller UFs when additional complexity is unnecessary. We show that our solution is Pareto-optimal against individual UFs, confirming the benefits of dynamic routing. When training the proposed dynamically-slimmable model to use 10% of its capacity on average, we obtain the same or better speech quality as the equivalent static 25% utilization while reducing MACs by 29%.

Key facts. Cited by 4 papers in the map (data through 7 Oct 2026). Builds on 32 of its 43 references that are in the map. First posted on arXiv 7 Jul 2025.

Builds on

32 of 43 references.

References · 32

  1. 2015U-Net: Convolutional Networks for Biomedical Image Segmentationcited by 19,604
  2. 2017Outrageously Large Neural Networks: The Sparsely-Gated Mixture-of-Experts Layercited by 3,413
  3. 2021Switch Transformers: Scaling to Trillion Parameter Models with Simple and Efficient Sparsitycited by 3,014
  4. 2016Categorical Reparameterization with Gumbel-Softmaxcited by 2,708
  5. 2016The Concrete Distribution: A Continuous Relaxation of Discrete Random Variablescited by 1,135
  6. 2016Language Modeling with Gated Convolutional Networkscited by 688
  7. 2018SDR - half-baked or well done?cited by 420
  8. 2021Dynamic Neural Networks: A Surveycited by 405
  9. 2018Slimmable Neural Networkscited by 301
  10. 2020DNSMOS: A Non-Intrusive Perceptual Objective Speech Quality metric to evaluate Noise Suppressorscited by 235
  11. 2020Real Time Speech Enhancement in the Waveform Domaincited by 183
  12. 2018Dynamic Channel Pruning: Feature Boosting and Suppressioncited by 129
20 more
  1. 2018Wave-U-Net: A Multi-Scale Neural Network for End-to-End Audio Source Separationcited by 125
  2. 2021Dynamic Slimmable Networkcited by 83
  3. 2020Why should we add early exits to neural networks?cited by 70
  4. 2019Demucs: Deep Extractor for Music Sources with extra unlabeled data remixedcited by 53
  5. 2020TinyLSTMs: Efficient Neural Speech Enhancement for Hearing Aidscited by 41
  6. 2021Towards efficient models for real-time deep noise suppressioncited by 28
  7. 2020A Perceptually-Motivated Approach for Low-Complexity, Real-Time Enhancement of Fullband Speechcited by 23
  8. 2020Data augmentation and loss normalization for deep noise suppressioncited by 23
  9. 2020Don't shoot butterfly with rifles: Multi-channel Continuous Speech Separation with Early Exit Transformercited by 15
  10. 2020Sparse Mixture of Local Experts for Efficient Speech Enhancementcited by 8
  11. 2022Accelerating RNN-based Speech Enhancement on a Multi-Core MCU with Mixed FP16-INT8 Post-Training Quantizationcited by 8
  12. 2019Rethinking Full Connectivity in Recurrent Neural Networkscited by 6
  13. 2022Latent Iterative Refinement for Modular Source Separationcited by 6
  14. 2023Dynamic nsNet2: Efficient Deep Noise Suppression with Early Exitingcited by 6
  15. 2020Speech Enhancement with Zero-Shot Model Selectioncited by 5
  16. 2021Learning to Inference with Early Exit in the Progressive Speech Enhancementcited by 5
  17. 2024Scalable Speech Enhancement with Dynamic Channel Pruningcited by 5
  18. 2021BLOOM-Net: Blockwise Optimization for Masking Networks Toward Scalable and Efficient Speech Enhancementcited by 3
  19. 2017Diagonal RNNs in Symbolic Music Modelingcited by 2
  20. 2023EffCRN: An Efficient Convolutional Recurrent Network for High-Performance Speech Enhancementcited by 2

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2026: 4.

4 papers.

  1. 2026From Diet to Free Lunch: Estimating Auxiliary Signal Properties using Dynamic Pruning Masks in Speech Enhancement Networkscited by 2
  2. 2026Performance and Complexity Trade-off Optimization of Speech Models During Trainingcited by 0
  3. 2026HybridSB-MoE: Dual-Domain Schr"odinger Bridges with Scene-Adaptive Expert Routing for Speech Enhancementcited by 0
  4. 2026Does per-frame early exit pay? A compute-matched study of dynamic depth for on-device speech enhancementcited by 0