# Real-time Controllable Denoising for Image and Video

arXiv: https://arxiv.org/abs/2303.16425 · HTML: https://mlsanity.com/paper/2303.16425/

## Key facts

- Cited by 9 papers in the map (data through 14 Oct 2026).
- Builds on 34 of its 65 references that are in the map.
- First posted on arXiv 29 Mar 2023.

## Builds on (most cited first, 30 of 34)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [Restormer: Efficient Transformer for High-Resolution Image Restoration](https://mlsanity.com/paper/2111.09881/) (2021)
- [Beyond a Gaussian Denoiser: Residual Learning of Deep CNN for Image Denoising](https://mlsanity.com/paper/1608.03981/) (2016)
- [Simple Baselines for Image Restoration](https://mlsanity.com/paper/2204.04676/) (2022)
- [Rethinking the Value of Network Pruning](https://mlsanity.com/paper/1810.05270/) (2018)
- [Generative Adversarial Networks: An Overview](https://mlsanity.com/paper/1710.07035/) (2017)
- [The Perception-Distortion Tradeoff](https://mlsanity.com/paper/1711.06077/) (2017)
- [Importance Estimation for Neural Network Pruning](https://mlsanity.com/paper/1906.10771/) (2019)
- [FFDNet: Toward a Fast and Flexible Solution for CNN based Image Denoising](https://mlsanity.com/paper/1710.04026/) (2017)
- [Learning Deep CNN Denoiser Prior for Image Restoration](https://mlsanity.com/paper/1704.03264/) (2017)
- [Toward Convolutional Blind Denoising of Real Photographs](https://mlsanity.com/paper/1807.04686/) (2018)
- [VRT: A Video Restoration Transformer](https://mlsanity.com/paper/2201.12288/) (2022)
- [Real Image Denoising with Feature Attention](https://mlsanity.com/paper/1904.07396/) (2019)
- [FastDVDnet: Towards Real-Time Deep Video Denoising Without Flow Estimation](https://mlsanity.com/paper/1907.01361/) (2019)
- [Iterative Normalization: Beyond Standardization towards Efficient Whitening](https://mlsanity.com/paper/1904.03441/) (2019)
- [Decorrelated Batch Normalization](https://mlsanity.com/paper/1804.08450/) (2018)
- [Real-world Noisy Image Denoising: A New Benchmark](https://mlsanity.com/paper/1804.02603/) (2018)
- [Deep Network Interpolation for Continuous Imagery Effect Transition](https://mlsanity.com/paper/1811.10515/) (2018)
- [Unsupervised Deep Video Denoising](https://mlsanity.com/paper/2011.15045/) (2020)
- [Modulating Image Restoration with Continual Levels via Adaptive Feature Modification Layers](https://mlsanity.com/paper/1904.08118/) (2019)
- [Patch Craft: Video Denoising by Deep Modeling and Patch Matching](https://mlsanity.com/paper/2103.13767/) (2021)
- [MM-RealSR: Metric Learning based Interactive Modulation for Real-World Super-Resolution](https://mlsanity.com/paper/2205.05065/) (2022)
- [Learning to Generate Realistic Noisy Images via Pixel-level Noise-aware Adversarial Training](https://mlsanity.com/paper/2204.02844/) (2022)
- [CFSNet: Toward a Controllable Feature Space for Image Restoration](https://mlsanity.com/paper/1904.00634/) (2019)
- [Differentiable Dynamic Quantization with Mixed Precision and Adaptive Resolution](https://mlsanity.com/paper/2106.02295/) (2021)
- [Toward Interactive Modulation for Photo-Realistic Image Restoration](https://mlsanity.com/paper/2105.03085/) (2021)
- [BLADE: Filter Learning for General Purpose Computational Photography](https://mlsanity.com/paper/1711.10700/) (2017)
- [Decouple Learning for Parameterized Image Operators](https://mlsanity.com/paper/1807.08186/) (2018)

## Built on by (most cited first, 9 of 9)

- [AutoDIR: Automatic All-in-One Image Restoration with Latent Diffusion](https://mlsanity.com/paper/2310.10123/) (2023)
- [Visual-Instructed Degradation Diffusion for All-in-One Image Restoration](https://mlsanity.com/paper/2506.16960/) (2025)
- [Prompt-based Ingredient-Oriented All-in-One Image Restoration](https://mlsanity.com/paper/2309.03063/) (2023)
- [Degradation-Aware Residual-Conditioned Optimal Transport for Unified Image Restoration](https://mlsanity.com/paper/2411.01656/) (2024)
- [Residual-Conditioned Optimal Transport: Towards Structure-Preserving Unpaired and Paired Image Restoration](https://mlsanity.com/paper/2405.02843/) (2024)
- [DualDn: Dual-domain Denoising via Differentiable ISP](https://mlsanity.com/paper/2409.18783/) (2024)
- [Make Explicit Calibration Implicit: Calibrate Denoiser Instead of the Noise Model](https://mlsanity.com/paper/2308.03448/) (2023)
- [Unsupervised Microscopy Video Denoising](https://mlsanity.com/paper/2404.12163/) (2024)
- [QueryCDR: Query-Based Controllable Distortion Rectification Network for Fisheye Images](https://mlsanity.com/paper/2412.13496/) (2024)

## Abstract (arXiv)

Controllable image denoising aims to generate clean samples with human perceptual priors and balance sharpness and smoothness. In traditional filter-based denoising methods, this can be easily achieved by adjusting the filtering strength. However, for NN (Neural Network)-based models, adjusting the final denoising strength requires performing network inference each time, making it almost impossible for real-time user interaction. In this paper, we introduce Real-time Controllable Denoising (RCD), the first deep image and video denoising pipeline that provides a fully controllable user interface to edit arbitrary denoising levels in real-time with only one-time network inference. Unlike existing controllable denoising methods that require multiple denoisers and training stages, RCD replaces the last output layer (which usually outputs a single noise map) of an existing CNN-based model with a lightweight module that outputs multiple noise maps. We propose a novel Noise Decorrelation process to enforce the orthogonality of the noise feature maps, allowing arbitrary noise level control through noise map interpolation. This process is network-free and does not require network inference. Our experiments show that RCD can enable real-time editable image and video denoising for various existing heavy-weight models without sacrificing their original performance.