# Dynamic Slimmable Denoising Network

arXiv: https://arxiv.org/abs/2110.08940 · HTML: https://mlsanity.com/paper/2110.08940/

## Key facts

- Cited by 9 papers in the map (data through 14 Oct 2026).
- Builds on 45 of its 62 references that are in the map.
- First posted on arXiv 17 Oct 2021.

## Builds on (most cited first, 30 of 45)

- [Distilling the Knowledge in a Neural Network](https://mlsanity.com/paper/1503.02531/) (2015)
- [EfficientNet: Rethinking Model Scaling for Convolutional Neural Networks](https://mlsanity.com/paper/1905.11946/) (2019)
- [MobileNetV2: Inverted Residuals and Linear Bottlenecks](https://mlsanity.com/paper/1801.04381/) (2018)
- [Squeeze-and-Excitation Networks](https://mlsanity.com/paper/1709.01507/) (2017)
- [MobileNets: Efficient Convolutional Neural Networks for Mobile Vision Applications](https://mlsanity.com/paper/1704.04861/) (2017)
- [Wide Residual Networks](https://mlsanity.com/paper/1605.07146/) (2016)
- [Quantization and Training of Neural Networks for Efficient Integer-Arithmetic-Only Inference](https://mlsanity.com/paper/1712.05877/) (2017)
- [Beyond a Gaussian Denoiser: Residual Learning of Deep CNN for Image Denoising](https://mlsanity.com/paper/1608.03981/) (2016)
- [Paying More Attention to Attention: Improving the Performance of Convolutional Neural Networks via Attention Transfer](https://mlsanity.com/paper/1612.03928/) (2016)
- [Multi-Stage Progressive Image Restoration](https://mlsanity.com/paper/2102.02808/) (2021)
- [Channel Pruning for Accelerating Very Deep Neural Networks](https://mlsanity.com/paper/1707.06168/) (2017)
- [FFDNet: Toward a Fast and Flexible Solution for CNN based Image Denoising](https://mlsanity.com/paper/1710.04026/) (2017)
- [Mean teachers are better role models: Weight-averaged consistency targets improve semi-supervised deep learning results](https://mlsanity.com/paper/1703.01780/) (2017)
- [Dynamic Neural Networks: A Survey](https://mlsanity.com/paper/2102.04906/) (2021)
- [SkipNet: Learning Dynamic Routing in Convolutional Networks](https://mlsanity.com/paper/1711.09485/) (2017)
- [Soft Filter Pruning for Accelerating Deep Convolutional Neural Networks](https://mlsanity.com/paper/1808.06866/) (2018)
- [Learning Enriched Features for Real Image Restoration and Enhancement](https://mlsanity.com/paper/2003.06792/) (2020)
- [Slimmable Neural Networks](https://mlsanity.com/paper/1812.08928/) (2018)
- [Toward Convolutional Blind Denoising of Real Photographs](https://mlsanity.com/paper/1807.04686/) (2018)
- [Progressive Image Deraining Networks: A Better and Simpler Baseline](https://mlsanity.com/paper/1901.09221/) (2019)
- [DynaBERT: Dynamic BERT with Adaptive Width and Depth](https://mlsanity.com/paper/2004.04037/) (2020)
- [MemNet: A Persistent Memory Network for Image Restoration](https://mlsanity.com/paper/1708.02209/) (2017)
- [MetaPruning: Meta Learning for Automatic Neural Network Channel Pruning](https://mlsanity.com/paper/1903.10258/) (2019)
- [Universally Slimmable Networks and Improved Training Techniques](https://mlsanity.com/paper/1903.05134/) (2019)
- [Benchmarking Denoising Algorithms with Real Photographs](https://mlsanity.com/paper/1707.01313/) (2017)
- [Deep Stacked Hierarchical Multi-patch Network for Image Deblurring](https://mlsanity.com/paper/1904.03468/) (2019)
- [Multi-level Wavelet-CNN for Image Restoration](https://mlsanity.com/paper/1805.07071/) (2018)
- [Real Image Denoising with Feature Attention](https://mlsanity.com/paper/1904.07396/) (2019)
- [Adaptive Neural Networks for Efficient Inference](https://mlsanity.com/paper/1702.07811/) (2017)
- [Non-Local Recurrent Network for Image Restoration](https://mlsanity.com/paper/1806.02919/) (2018)

## Built on by (most cited first, 9 of 9)

- [Efficient Training of Large Vision Models via Advanced Automated Progressive Learning](https://mlsanity.com/paper/2410.00350/) (2024)
- [SlimSeg: Slimmable Semantic Segmentation with Boundary Supervision](https://mlsanity.com/paper/2207.06242/) (2022)
- [DNA Family: Boosting Weight-Sharing NAS with Block-Wise Supervisions](https://mlsanity.com/paper/2403.01326/) (2024)
- [NaviSlim: Adaptive Context-Aware Navigation and Sensing via Dynamic Slimmable Networks](https://mlsanity.com/paper/2407.01563/) (2024)
- [Regret Minimization in Population Network Games: Vanishing Heterogeneity and Convergence to Equilibria](https://mlsanity.com/paper/2507.17183/) (2025)
- [Powerful Lossy Compression for Noisy Images](https://mlsanity.com/paper/2403.14135/) (2024)
- [Task-Aware Dynamic Transformer for Efficient Arbitrary-Scale Image Super-Resolution](https://mlsanity.com/paper/2408.08736/) (2024)
- [Adaptive Guided Upsampling for Low-light Image Enhancement](https://mlsanity.com/paper/2511.16623/) (2025)
- [CADENCE: Context-Adaptive Depth Estimation for Navigation and Computational Efficiency](https://mlsanity.com/paper/2604.07286/) (2026)

## Abstract (arXiv)

Recently, tremendous human-designed and automatically searched neural networks have been applied to image denoising. However, previous works intend to handle all noisy images in a pre-defined static network architecture, which inevitably leads to high computational complexity for good denoising quality. Here, we present dynamic slimmable denoising network (DDS-Net), a general method to achieve good denoising quality with less computational complexity, via dynamically adjusting the channel configurations of networks at test time with respect to different noisy images. Our DDS-Net is empowered with the ability of dynamic inference by a dynamic gate, which can predictively adjust the channel configuration of networks with negligible extra computation cost. To ensure the performance of each candidate sub-network and the fairness of the dynamic gate, we propose a three-stage optimization scheme. In the first stage, we train a weight-shared slimmable super network. In the second stage, we evaluate the trained slimmable super network in an iterative way and progressively tailor the channel numbers of each layer with minimal denoising quality drop. By a single pass, we can obtain several sub-networks with good performance under different channel configurations. In the last stage, we identify easy and hard samples in an online way and train a dynamic gate to predictively select the corresponding sub-network with respect to different noisy images. Extensive experiments demonstrate our DDS-Net consistently outperforms the state-of-the-art individually trained static denoising networks.