# Adaptive Denoising via GainTuning

arXiv: https://arxiv.org/abs/2107.12815 · HTML: https://mlsanity.com/paper/2107.12815/

## Key facts

- Cited by 16 papers in the map (data through 14 Oct 2026).
- Builds on 31 of its 83 references that are in the map.
- First posted on arXiv 27 Jul 2021.

## Builds on (most cited first, 30 of 31)

- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [U-Net: Convolutional Networks for Biomedical Image Segmentation](https://mlsanity.com/paper/1505.04597/) (2015)
- [Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shift](https://mlsanity.com/paper/1502.03167/) (2015)
- [FiLM: Visual Reasoning with a General Conditioning Layer](https://mlsanity.com/paper/1709.07871/) (2017)
- [Beyond a Gaussian Denoiser: Residual Learning of Deep CNN for Image Denoising](https://mlsanity.com/paper/1608.03981/) (2016)
- [How transferable are features in deep neural networks?](https://mlsanity.com/paper/1411.1792/) (2014)
- [End-to-end Optimized Image Compression](https://mlsanity.com/paper/1611.01704/) (2016)
- [FFDNet: Toward a Fast and Flexible Solution for CNN based Image Denoising](https://mlsanity.com/paper/1710.04026/) (2017)
- [Improving robustness against common corruptions by covariate shift adaptation](https://mlsanity.com/paper/2006.16971/) (2020)
- [DeCAF: A Deep Convolutional Activation Feature for Generic Visual Recognition](https://mlsanity.com/paper/1310.1531/) (2013)
- [Noise2Void - Learning Denoising from Single Noisy Images](https://mlsanity.com/paper/1811.10980/) (2018)
- [Noise2Self: Blind Denoising by Self-Supervision](https://mlsanity.com/paper/1901.11365/) (2019)
- [Evaluating Prediction-Time Batch Normalization for Robustness under Covariate Shift](https://mlsanity.com/paper/2006.10963/) (2020)
- [Trainable Nonlinear Reaction Diffusion: A Flexible Framework for Fast and Effective Image Restoration](https://mlsanity.com/paper/1508.02848/) (2015)
- [Modulating early visual processing by language](https://mlsanity.com/paper/1707.00683/) (2017)
- [High-Quality Self-Supervised Deep Image Denoising](https://mlsanity.com/paper/1901.10277/) (2019)
- [Density Modeling of Images using a Generalized Normalization Transformation](https://mlsanity.com/paper/1511.06281/) (2015)
- [Noisier2Noise: Learning to Denoise from Unpaired Noisy Data](https://mlsanity.com/paper/1910.11908/) (2019)
- [Robust and interpretable blind image denoising via bias-free convolutional neural networks](https://mlsanity.com/paper/1906.05478/) (2019)
- [Exploring the structure of a real-time, arbitrary neural artistic stylization network](https://mlsanity.com/paper/1705.06830/) (2017)
- [Noise2Same: Optimizing A Self-Supervised Bound for Image Denoising](https://mlsanity.com/paper/2010.11971/) (2020)
- [Probabilistic Noise2Void: Unsupervised Content-Aware Denoising](https://mlsanity.com/paper/1906.00651/) (2019)
- [Unsupervised Learning with Stein's Unbiased Risk Estimator](https://mlsanity.com/paper/1805.10531/) (2018)
- [Unsupervised Deep Video Denoising](https://mlsanity.com/paper/2011.15045/) (2020)
- [Training Deep Learning Based Denoisers without Ground Truth Data](https://mlsanity.com/paper/1803.01314/) (2018)
- [Modulating Image Restoration with Continual Levels via Adaptive Feature Modification Layers](https://mlsanity.com/paper/1904.08118/) (2019)
- [On Self Modulation for Generative Adversarial Networks](https://mlsanity.com/paper/1810.01365/) (2018)
- [Deep Denoising For Scientific Discovery: A Case Study In Electron Microscopy](https://mlsanity.com/paper/2010.12970/) (2020)
- [Developing and Evaluating Deep Neural Network-based Denoising for Nanoparticle TEM Images with Ultra-low Signal-to-Noise](https://mlsanity.com/paper/2101.07770/) (2021)
- [Dynamically Unfolding Recurrent Restorer: A Moving Endpoint Control Method for Image Restoration](https://mlsanity.com/paper/1805.07709/) (2018)

## Built on by (most cited first, 16 of 16)

- [A Comprehensive Survey on Test-Time Adaptation under Distribution Shifts](https://mlsanity.com/paper/2303.15361/) (2023)
- [Transfer CLIP for Generalizable Image Denoising](https://mlsanity.com/paper/2403.15132/) (2024)
- [Joint Video Multi-Frame Interpolation and Deblurring under Unknown Exposure Time](https://mlsanity.com/paper/2303.15043/) (2023)
- [IDF: Iterative Dynamic Filtering Networks for Generalizable Image Denoising](https://mlsanity.com/paper/2508.19649/) (2025)
- [Deep Internal Learning: Deep Learning from a Single Input](https://mlsanity.com/paper/2312.07425/) (2023)
- [Patch-Craft Self-Supervised Training for Correlated Image Denoising](https://mlsanity.com/paper/2211.09919/) (2022)
- [Plug-and-Play Versatile Compressed Video Enhancement](https://mlsanity.com/paper/2504.15380/) (2025)
- [Self-Supervised Learning from Noisy and Incomplete Data](https://mlsanity.com/paper/2601.03244/) (2026)
- [Linear Combinations of Patches are Unreasonably Effective for Single-Image Denoising](https://mlsanity.com/paper/2212.00422/) (2022)
- [Coordinate-Aware Modulation for Neural Fields](https://mlsanity.com/paper/2311.14993/) (2023)
- [Evaluating Unsupervised Denoising Requires Unsupervised Metrics](https://mlsanity.com/paper/2210.05553/) (2022)
- [Adaptive whitening in neural populations with gain-modulating interneurons](https://mlsanity.com/paper/2301.11955/) (2023)
- [Adaptive whitening with fast gain modulation and slow synaptic plasticity](https://mlsanity.com/paper/2308.13633/) (2023)
- [Diffscaler: Enhancing the Generative Prowess of Diffusion Transformers](https://mlsanity.com/paper/2404.09976/) (2024)
- [SelfReDepth: Self-Supervised Real-Time Depth Restoration for Consumer-Grade Sensors](https://mlsanity.com/paper/2406.03388/) (2024)
- [Atomic Depth Estimation From Noisy Electron Microscopy Data Via Deep Learning](https://mlsanity.com/paper/2601.17046/) (2026)

## Abstract (arXiv)

Deep convolutional neural networks (CNNs) for image denoising are usually trained on large datasets. These models achieve the current state of the art, but they have difficulties generalizing when applied to data that deviate from the training distribution. Recent work has shown that it is possible to train denoisers on a single noisy image. These models adapt to the features of the test image, but their performance is limited by the small amount of information used to train them. Here we propose "GainTuning", in which CNN models pre-trained on large datasets are adaptively and selectively adjusted for individual test images. To avoid overfitting, GainTuning optimizes a single multiplicative scaling parameter (the "Gain") of each channel in the convolutional layers of the CNN. We show that GainTuning improves state-of-the-art CNNs on standard image-denoising benchmarks, boosting their denoising performance on nearly every image in a held-out test set. These adaptive improvements are even more substantial for test images differing systematically from the training data, either in noise level or image type. We illustrate the potential of adaptive denoising in a scientific application, in which a CNN is trained on synthetic data, and tested on real transmission-electron-microscope images. In contrast to the existing methodology, GainTuning is able to faithfully reconstruct the structure of catalytic nanoparticles from these data at extremely low signal-to-noise ratios.