# NM-FlowGAN: Modeling sRGB Noise without Paired Images using a Hybrid Approach of Normalizing Flows and GAN

arXiv: https://arxiv.org/abs/2312.10112 · HTML: https://mlsanity.com/paper/2312.10112/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 25 of its 37 references that are in the map.
- First posted on arXiv 15 Dec 2023.

## Builds on (most cited first, 25 of 25)

- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [U-Net: Convolutional Networks for Biomedical Image Segmentation](https://mlsanity.com/paper/1505.04597/) (2015)
- [Very Deep Convolutional Networks for Large-Scale Image Recognition](https://mlsanity.com/paper/1409.1556/) (2014)
- [Generative Adversarial Networks](https://mlsanity.com/paper/2203.00667/) (2022)
- [Exploring Simple Siamese Representation Learning](https://mlsanity.com/paper/2011.10566/) (2020)
- [Improved Training of Wasserstein GANs](https://mlsanity.com/paper/1704.00028/) (2017)
- [Variational Inference with Normalizing Flows](https://mlsanity.com/paper/1505.05770/) (2015)
- [Density estimation using Real NVP](https://mlsanity.com/paper/1605.08803/) (2016)
- [Glow: Generative Flow with Invertible 1x1 Convolutions](https://mlsanity.com/paper/1807.03039/) (2018)
- [Beyond a Gaussian Denoiser: Residual Learning of Deep CNN for Image Denoising](https://mlsanity.com/paper/1608.03981/) (2016)
- [Normalizing Flows for Probabilistic Modeling and Inference](https://mlsanity.com/paper/1912.02762/) (2019)
- [Toward Convolutional Blind Denoising of Real Photographs](https://mlsanity.com/paper/1807.04686/) (2018)
- [Noise Flow: Noise Modeling with Conditional Normalizing Flows](https://mlsanity.com/paper/1908.08453/) (2019)
- [AP-BSN: Self-Supervised Denoising for Real-World Images via Asymmetric PD and Blind-Spot Network](https://mlsanity.com/paper/2203.11799/) (2022)
- [Real-world Noisy Image Denoising: A New Benchmark](https://mlsanity.com/paper/1804.02603/) (2018)
- [C2N: Practical Generative Noise Modeling for Real-World Denoising](https://mlsanity.com/paper/2202.09533/) (2022)
- [When AWGN-based Denoiser Meets Real Noises](https://mlsanity.com/paper/1904.03485/) (2019)
- [NTIRE 2020 Challenge on Real Image Denoising: Dataset, Methods and Results](https://mlsanity.com/paper/2005.04117/) (2020)
- [Learning Camera-Aware Noise Models](https://mlsanity.com/paper/2008.09370/) (2020)
- [Modeling sRGB Camera Noise with Normalizing Flows](https://mlsanity.com/paper/2206.00812/) (2022)
- [Natural Image Noise Dataset](https://mlsanity.com/paper/1906.00270/) (2019)
- [Noise2NoiseFlow: Realistic Camera Noise Modeling without Clean Images](https://mlsanity.com/paper/2206.01103/) (2022)
- [A Review of an Old Dilemma: Demosaicking First, or Denoising First?](https://mlsanity.com/paper/2004.11577/) (2020)
- [NoiseTransfer: Image Noise Generation with Contrastive Embeddings](https://mlsanity.com/paper/2301.13554/) (2023)
- [A Generative Model for Digital Camera Noise Synthesis](https://mlsanity.com/paper/2303.09199/) (2023)

## Abstract (arXiv)

Modeling and synthesizing real sRGB noise is crucial for various low-level vision tasks, such as building datasets for training image denoising systems. The distribution of real sRGB noise is highly complex and affected by a multitude of factors, making its accurate modeling extremely challenging. Therefore, recent studies have proposed methods that employ data-driven generative models, such as Generative Adversarial Networks (GAN) and Normalizing Flows. These studies achieve more accurate modeling of sRGB noise compared to traditional noise modeling methods. However, there are performance limitations due to the inherent characteristics of each generative model. To address this issue, we propose NM-FlowGAN, a hybrid approach that exploits the strengths of both GAN and Normalizing Flows. We combine pixel-wise noise modeling networks based on Normalizing Flows and spatial correlation modeling networks based on GAN. Specifically, the pixel-wise noise modeling network leverages the high training stability of Normalizing Flows to capture noise characteristics that are affected by a multitude of factors, and the spatial correlation networks efficiently model pixel-to-pixel relationships. In particular, unlike recent methods that rely on paired noisy images, our method synthesizes noise using clean images and factors that affect noise characteristics, such as easily obtainable parameters like camera type and ISO settings, making it applicable to various fields where obtaining noisy-clean image pairs is not feasible. In our experiments, our NM-FlowGAN outperforms other baselines in the sRGB noise synthesis task. Moreover, the denoising neural network trained with synthesized image pairs from our model shows superior performance compared to other baselines. Our code is available at: urlhttps://github.com/YoungJooHan/NM-FlowGAN.