# Adaptive Slimming for Scalable and Efficient Speech Enhancement

arXiv: https://arxiv.org/abs/2507.04879 · HTML: https://mlsanity.com/paper/2507.04879/

## Key facts

- Cited by 4 papers in the map (data through 14 Oct 2026).
- Builds on 32 of its 43 references that are in the map.
- First posted on arXiv 7 Jul 2025.

## Builds on (most cited first, 30 of 32)

- [U-Net: Convolutional Networks for Biomedical Image Segmentation](https://mlsanity.com/paper/1505.04597/) (2015)
- [Outrageously Large Neural Networks: The Sparsely-Gated Mixture-of-Experts Layer](https://mlsanity.com/paper/1701.06538/) (2017)
- [Switch Transformers: Scaling to Trillion Parameter Models with Simple and Efficient Sparsity](https://mlsanity.com/paper/2101.03961/) (2021)
- [Categorical Reparameterization with Gumbel-Softmax](https://mlsanity.com/paper/1611.01144/) (2016)
- [The Concrete Distribution: A Continuous Relaxation of Discrete Random Variables](https://mlsanity.com/paper/1611.00712/) (2016)
- [Language Modeling with Gated Convolutional Networks](https://mlsanity.com/paper/1612.08083/) (2016)
- [SDR - half-baked or well done?](https://mlsanity.com/paper/1811.02508/) (2018)
- [Dynamic Neural Networks: A Survey](https://mlsanity.com/paper/2102.04906/) (2021)
- [Slimmable Neural Networks](https://mlsanity.com/paper/1812.08928/) (2018)
- [DNSMOS: A Non-Intrusive Perceptual Objective Speech Quality metric to evaluate Noise Suppressors](https://mlsanity.com/paper/2010.15258/) (2020)
- [Real Time Speech Enhancement in the Waveform Domain](https://mlsanity.com/paper/2006.12847/) (2020)
- [Dynamic Channel Pruning: Feature Boosting and Suppression](https://mlsanity.com/paper/1810.05331/) (2018)
- [Wave-U-Net: A Multi-Scale Neural Network for End-to-End Audio Source Separation](https://mlsanity.com/paper/1806.03185/) (2018)
- [Dynamic Slimmable Network](https://mlsanity.com/paper/2103.13258/) (2021)
- [Why should we add early exits to neural networks?](https://mlsanity.com/paper/2004.12814/) (2020)
- [Demucs: Deep Extractor for Music Sources with extra unlabeled data remixed](https://mlsanity.com/paper/1909.01174/) (2019)
- [TinyLSTMs: Efficient Neural Speech Enhancement for Hearing Aids](https://mlsanity.com/paper/2005.11138/) (2020)
- [Towards efficient models for real-time deep noise suppression](https://mlsanity.com/paper/2101.09249/) (2021)
- [A Perceptually-Motivated Approach for Low-Complexity, Real-Time Enhancement of Fullband Speech](https://mlsanity.com/paper/2008.04259/) (2020)
- [Data augmentation and loss normalization for deep noise suppression](https://mlsanity.com/paper/2008.06412/) (2020)
- [Don't shoot butterfly with rifles: Multi-channel Continuous Speech Separation with Early Exit Transformer](https://mlsanity.com/paper/2010.12180/) (2020)
- [Sparse Mixture of Local Experts for Efficient Speech Enhancement](https://mlsanity.com/paper/2005.08128/) (2020)
- [Accelerating RNN-based Speech Enhancement on a Multi-Core MCU with Mixed FP16-INT8 Post-Training Quantization](https://mlsanity.com/paper/2210.07692/) (2022)
- [Latent Iterative Refinement for Modular Source Separation](https://mlsanity.com/paper/2211.11917/) (2022)
- [Dynamic nsNet2: Efficient Deep Noise Suppression with Early Exiting](https://mlsanity.com/paper/2308.16678/) (2023)
- [Rethinking Full Connectivity in Recurrent Neural Networks](https://mlsanity.com/paper/1905.12340/) (2019)
- [Learning to Inference with Early Exit in the Progressive Speech Enhancement](https://mlsanity.com/paper/2106.11730/) (2021)
- [Speech Enhancement with Zero-Shot Model Selection](https://mlsanity.com/paper/2012.09359/) (2020)
- [Scalable Speech Enhancement with Dynamic Channel Pruning](https://mlsanity.com/paper/2412.17121/) (2024)
- [BLOOM-Net: Blockwise Optimization for Masking Networks Toward Scalable and Efficient Speech Enhancement](https://mlsanity.com/paper/2111.09372/) (2021)

## Built on by (most cited first, 4 of 4)

- [From Diet to Free Lunch: Estimating Auxiliary Signal Properties using Dynamic Pruning Masks in Speech Enhancement Networks](https://mlsanity.com/paper/2602.10666/) (2026)
- [Performance and Complexity Trade-off Optimization of Speech Models During Training](https://mlsanity.com/paper/2601.13704/) (2026)
- [HybridSB-MoE: Dual-Domain Schr"odinger Bridges with Scene-Adaptive Expert Routing for Speech Enhancement](https://mlsanity.com/paper/2608.12715/) (2026)
- [Does per-frame early exit pay? A compute-matched study of dynamic depth for on-device speech enhancement](https://mlsanity.com/paper/2609.29867/) (2026)

## Abstract (arXiv)

Speech enhancement (SE) enables robust speech recognition, real-time communication, hearing aids, and other applications where speech quality is crucial. However, deploying such systems on resource-constrained devices involves choosing a static trade-off between performance and computational efficiency. In this paper, we introduce dynamic slimming to DEMUCS, a popular SE architecture, making it scalable and input-adaptive. Slimming lets the model operate at different utilization factors (UF), each corresponding to a different performance/efficiency trade-off, effectively mimicking multiple model sizes without the extra storage costs. In addition, a router subnet, trained end-to-end with the backbone, determines the optimal UF for the current input. Thus, the system saves resources by adaptively selecting smaller UFs when additional complexity is unnecessary. We show that our solution is Pareto-optimal against individual UFs, confirming the benefits of dynamic routing. When training the proposed dynamically-slimmable model to use 10% of its capacity on average, we obtain the same or better speech quality as the equivalent static 25% utilization while reducing MACs by 29%.