# A Generative Model for Digital Camera Noise Synthesis

arXiv: https://arxiv.org/abs/2303.09199 · HTML: https://mlsanity.com/paper/2303.09199/

## Key facts

- Cited by 3 papers in the map (data through 14 Oct 2026).
- Builds on 27 of its 31 references that are in the map.
- First posted on arXiv 16 Mar 2023.

## Builds on (most cited first, 27 of 27)

- [Very Deep Convolutional Networks for Large-Scale Image Recognition](https://mlsanity.com/paper/1409.1556/) (2014)
- [Layer Normalization](https://mlsanity.com/paper/1607.06450/) (2016)
- [Generative Adversarial Networks](https://mlsanity.com/paper/2203.00667/) (2022)
- [SGDR: Stochastic Gradient Descent with Warm Restarts](https://mlsanity.com/paper/1608.03983/) (2016)
- [Squeeze-and-Excitation Networks](https://mlsanity.com/paper/1709.01507/) (2017)
- [Analyzing and Improving the Image Quality of StyleGAN](https://mlsanity.com/paper/1912.04958/) (2019)
- [Variational Inference with Normalizing Flows](https://mlsanity.com/paper/1505.05770/) (2015)
- [SwinIR: Image Restoration Using Swin Transformer](https://mlsanity.com/paper/2108.10257/) (2021)
- [GLU Variants Improve Transformer](https://mlsanity.com/paper/2002.05202/) (2020)
- [Restormer: Efficient Transformer for High-Resolution Image Restoration](https://mlsanity.com/paper/2111.09881/) (2021)
- [Beyond a Gaussian Denoiser: Residual Learning of Deep CNN for Image Denoising](https://mlsanity.com/paper/1608.03981/) (2016)
- [Simple Baselines for Image Restoration](https://mlsanity.com/paper/2204.04676/) (2022)
- [Noise2Noise: Learning Image Restoration without Clean Data](https://mlsanity.com/paper/1803.04189/) (2018)
- [Demystifying Neural Style Transfer](https://mlsanity.com/paper/1701.01036/) (2017)
- [A Physics-based Noise Formation Model for Extreme Low-light Raw Denoising](https://mlsanity.com/paper/2003.12751/) (2020)
- [FastDVDnet: Towards Real-Time Deep Video Denoising Without Flow Estimation](https://mlsanity.com/paper/1907.01361/) (2019)
- [Noise Flow: Noise Modeling with Conditional Normalizing Flows](https://mlsanity.com/paper/1908.08453/) (2019)
- [Dual Adversarial Network: Toward Real-world Noise Removal and Noise Generation](https://mlsanity.com/paper/2007.05946/) (2020)
- [Invertible Denoising Network: A Light Solution for Real Noise Removal](https://mlsanity.com/paper/2104.10546/) (2021)
- [Rethinking Noise Synthesis and Modeling in Raw Denoising](https://mlsanity.com/paper/2110.04756/) (2021)
- [C2N: Practical Generative Noise Modeling for Real-World Denoising](https://mlsanity.com/paper/2202.09533/) (2022)
- [Dancing under the stars: video denoising in starlight](https://mlsanity.com/paper/2204.04210/) (2022)
- [Learning to Generate Realistic Noisy Images via Pixel-level Noise-aware Adversarial Training](https://mlsanity.com/paper/2204.02844/) (2022)
- [Modeling sRGB Camera Noise with Normalizing Flows](https://mlsanity.com/paper/2206.00812/) (2022)
- [Noise2NoiseFlow: Realistic Camera Noise Modeling without Clean Images](https://mlsanity.com/paper/2206.01103/) (2022)
- [Estimating Fine-Grained Noise Model via Contrastive Learning](https://mlsanity.com/paper/2204.01716/) (2022)
- [Deep-based Film Grain Removal and Synthesis](https://mlsanity.com/paper/2206.07411/) (2022)

## Built on by (most cited first, 3 of 3)

- [Realistic Noise Synthesis with Diffusion Models](https://mlsanity.com/paper/2305.14022/) (2023)
- [Revitalizing Legacy Video Content: Deinterlacing with Bidirectional Information Propagation](https://mlsanity.com/paper/2310.19535/) (2023)
- [NM-FlowGAN: Modeling sRGB Noise without Paired Images using a Hybrid Approach of Normalizing Flows and GAN](https://mlsanity.com/paper/2312.10112/) (2023)

## Abstract (arXiv)

Noise synthesis is a challenging low-level vision task aiming to generate realistic noise given a clean image along with the camera settings. To this end, we propose an effective generative model which utilizes clean features as guidance followed by noise injections into the network. Specifically, our generator follows a UNet-like structure with skip connections but without downsampling and upsampling layers. Firstly, we extract deep features from a clean image as the guidance and concatenate a Gaussian noise map to the transition point between the encoder and decoder as the noise source. Secondly, we propose noise synthesis blocks in the decoder in each of which we inject Gaussian noise to model the noise characteristics. Thirdly, we propose to utilize an additional Style Loss and demonstrate that this allows better noise characteristics supervision in the generator. Through a number of new experiments, we evaluate the temporal variance and the spatial correlation of the generated noise which we hope can provide meaningful insights for future works. Finally, we show that our proposed approach outperforms existing methods for synthesizing camera noise.