# Learning to Generate Realistic Noisy Images via Pixel-level Noise-aware Adversarial Training

arXiv: https://arxiv.org/abs/2204.02844 · HTML: https://mlsanity.com/paper/2204.02844/

## Key facts

- Cited by 29 papers in the map (data through 14 Oct 2026).
- Builds on 39 of its 65 references that are in the map.
- First posted on arXiv 6 Apr 2022.

## Builds on (most cited first, 30 of 39)

- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [Very Deep Convolutional Networks for Large-Scale Image Recognition](https://mlsanity.com/paper/1409.1556/) (2014)
- [ImageNet Large Scale Visual Recognition Challenge](https://mlsanity.com/paper/1409.0575/) (2014)
- [SGDR: Stochastic Gradient Descent with Warm Restarts](https://mlsanity.com/paper/1608.03983/) (2016)
- [Image-to-Image Translation with Conditional Adversarial Networks](https://mlsanity.com/paper/1611.07004/) (2016)
- [Photo-Realistic Single Image Super-Resolution Using a Generative Adversarial Network](https://mlsanity.com/paper/1609.04802/) (2016)
- [Enhanced Deep Residual Networks for Single Image Super-Resolution](https://mlsanity.com/paper/1707.02921/) (2017)
- [Beyond a Gaussian Denoiser: Residual Learning of Deep CNN for Image Denoising](https://mlsanity.com/paper/1608.03981/) (2016)
- [ESRGAN: Enhanced Super-Resolution Generative Adversarial Networks](https://mlsanity.com/paper/1809.00219/) (2018)
- [Multi-Stage Progressive Image Restoration](https://mlsanity.com/paper/2102.02808/) (2021)
- [Generative Image Inpainting with Contextual Attention](https://mlsanity.com/paper/1801.07892/) (2018)
- [FFDNet: Toward a Fast and Flexible Solution for CNN based Image Denoising](https://mlsanity.com/paper/1710.04026/) (2017)
- [EnlightenGAN: Deep Light Enhancement without Paired Supervision](https://mlsanity.com/paper/1906.06972/) (2019)
- [Learning Deep CNN Denoiser Prior for Image Restoration](https://mlsanity.com/paper/1704.03264/) (2017)
- [Making Convolutional Networks Shift-Invariant Again](https://mlsanity.com/paper/1904.11486/) (2019)
- [Learning Enriched Features for Real Image Restoration and Enhancement](https://mlsanity.com/paper/2003.06792/) (2020)
- [Toward Convolutional Blind Denoising of Real Photographs](https://mlsanity.com/paper/1807.04686/) (2018)
- [The relativistic discriminator: a key element missing from standard GAN](https://mlsanity.com/paper/1807.00734/) (2018)
- [CycleISP: Real Image Restoration via Improved Data Synthesis](https://mlsanity.com/paper/2003.07761/) (2020)
- [Benchmarking Denoising Algorithms with Real Photographs](https://mlsanity.com/paper/1707.01313/) (2017)
- [Attentive Generative Adversarial Network for Raindrop Removal from a Single Image](https://mlsanity.com/paper/1711.10098/) (2017)
- [Residual Dense Network for Image Restoration](https://mlsanity.com/paper/1812.10477/) (2018)
- [Real Image Denoising with Feature Attention](https://mlsanity.com/paper/1904.07396/) (2019)
- [Pluralistic Image Completion](https://mlsanity.com/paper/1903.04227/) (2019)
- [Unsupervised Image Super-Resolution using Cycle-in-Cycle Generative Adversarial Networks](https://mlsanity.com/paper/1809.00437/) (2018)
- [Variational Denoising Network: Toward Blind Noise Modeling and Removal](https://mlsanity.com/paper/1908.11314/) (2019)
- [Mask-guided Spectral-wise Transformer for Efficient Hyperspectral Image Reconstruction](https://mlsanity.com/paper/2111.07910/) (2021)
- [Combining Markov Random Fields and Convolutional Neural Networks for Image Synthesis](https://mlsanity.com/paper/1601.04589/) (2016)
- [Dual Adversarial Network: Toward Real-world Noise Removal and Noise Generation](https://mlsanity.com/paper/2007.05946/) (2020)
- [AutoGAN: Neural Architecture Search for Generative Adversarial Networks](https://mlsanity.com/paper/1908.03835/) (2019)

## Built on by (most cited first, 29 of 29)

- [MST++: Multi-stage Spectral-wise Transformer for Efficient Spectral Reconstruction](https://mlsanity.com/paper/2204.07908/) (2022)
- [Degradation-Aware Unfolding Half-Shuffle Transformer for Spectral Compressive Imaging](https://mlsanity.com/paper/2205.10102/) (2022)
- [Random Sub-Samples Generation for Self-Supervised Real Image Denoising](https://mlsanity.com/paper/2307.16825/) (2023)
- [Friendly Noise against Adversarial Noise: A Powerful Defense against Data Poisoning Attacks](https://mlsanity.com/paper/2208.10224/) (2022)
- [Detecting Generated Images by Real Images Only](https://mlsanity.com/paper/2311.00962/) (2023)
- [Towards General Low-Light Raw Noise Synthesis and Modeling](https://mlsanity.com/paper/2307.16508/) (2023)
- [Stimulating Diffusion Model for Image Denoising via Adaptive Embedding and Ensembling](https://mlsanity.com/paper/2307.03992/) (2023)
- [Self-supervised Image Denoising with Downsampled Invariance Loss and Conditional Blind-Spot Network](https://mlsanity.com/paper/2304.09507/) (2023)
- [Unsupervised Flow-Aligned Sequence-to-Sequence Learning for Video Restoration](https://mlsanity.com/paper/2205.10195/) (2022)
- [Learning Physics-Informed Noise Models from Dark Frames for Low-Light Raw Image Denoising](https://mlsanity.com/paper/2310.09126/) (2023)
- [Real-time Controllable Denoising for Image and Video](https://mlsanity.com/paper/2303.16425/) (2023)
- [Denoising as Adaptation: Noise-Space Domain Adaptation for Image Restoration](https://mlsanity.com/paper/2406.18516/) (2024)
- [Modular Degradation Simulation and Restoration for Under-Display Camera](https://mlsanity.com/paper/2209.11455/) (2022)
- [Denoising Diffusion Post-Processing for Low-Light Image Enhancement](https://mlsanity.com/paper/2303.09627/) (2023)
- [A Comparison of Image Denoising Methods](https://mlsanity.com/paper/2304.08990/) (2023)
- [Realistic Noise Synthesis with Diffusion Models](https://mlsanity.com/paper/2305.14022/) (2023)
- [NoiseTransfer: Image Noise Generation with Contrastive Embeddings](https://mlsanity.com/paper/2301.13554/) (2023)
- [GMSR:Gradient-Guided Mamba for Spectral Reconstruction from RGB Images](https://mlsanity.com/paper/2405.07777/) (2024)
- [Self-Calibrated Variance-Stabilizing Transformations for Real-World Image Denoising](https://mlsanity.com/paper/2407.17399/) (2024)
- [Image Denoising Using Green Channel Prior](https://mlsanity.com/paper/2408.05923/) (2024)
- [Learning to Translate Noise for Robust Image Denoising](https://mlsanity.com/paper/2412.04727/) (2024)
- [A Generative Model for Digital Camera Noise Synthesis](https://mlsanity.com/paper/2303.09199/) (2023)
- [Adaptive Sparse and Monotonic Attention for Transformer-based Automatic Speech Recognition](https://mlsanity.com/paper/2209.15176/) (2022)
- [TreEnhance: A Tree Search Method For Low-Light Image Enhancement](https://mlsanity.com/paper/2205.12639/) (2022)
- [On normalization-equivariance properties of supervised and unsupervised denoising methods: a survey](https://mlsanity.com/paper/2402.15352/) (2024)
- [FRN: Fractal-Based Recursive Spectral Reconstruction Network](https://mlsanity.com/paper/2505.15439/) (2025)
- [An Ensemble Model for Distorted Images in Real Scenarios](https://mlsanity.com/paper/2309.14998/) (2023)
- [GuidNoise: Single-Pair Guided Diffusion for Generalized Noise Synthesis](https://mlsanity.com/paper/2512.04456/) (2025)
- [sRGB Real Noise Modeling via Noise-Aware Sampling with Normalizing Flows](https://mlsanity.com/paper/2608.29038/) (2026)

## Abstract (arXiv)

Existing deep learning real denoising methods require a large amount of noisy-clean image pairs for supervision. Nonetheless, capturing a real noisy-clean dataset is an unacceptable expensive and cumbersome procedure. To alleviate this problem, this work investigates how to generate realistic noisy images. Firstly, we formulate a simple yet reasonable noise model that treats each real noisy pixel as a random variable. This model splits the noisy image generation problem into two sub-problems: image domain alignment and noise domain alignment. Subsequently, we propose a novel framework, namely Pixel-level Noise-aware Generative Adversarial Network (PNGAN). PNGAN employs a pre-trained real denoiser to map the fake and real noisy images into a nearly noise-free solution space to perform image domain alignment. Simultaneously, PNGAN establishes a pixel-level adversarial training to conduct noise domain alignment. Additionally, for better noise fitting, we present an efficient architecture Simple Multi-scale Network (SMNet) as the generator. Qualitative validation shows that noise generated by PNGAN is highly similar to real noise in terms of intensity and distribution. Quantitative experiments demonstrate that a series of denoisers trained with the generated noisy images achieve state-of-the-art (SOTA) results on four real denoising benchmarks. Part of codes, pre-trained models, and results are available at https://github.com/caiyuanhao1998/PNGAN for comparisons.