# FSID: Fully Synthetic Image Denoising via Procedural Scene Generation

arXiv: https://arxiv.org/abs/2212.03961 · HTML: https://mlsanity.com/paper/2212.03961/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 29 of its 43 references that are in the map.
- First posted on arXiv 7 Dec 2022.

## Builds on (most cited first, 29 of 29)

- [Microsoft COCO: Common Objects in Context](https://mlsanity.com/paper/1405.0312/) (2014)
- [ImageNet Large Scale Visual Recognition Challenge](https://mlsanity.com/paper/1409.0575/) (2014)
- [A Style-Based Generator Architecture for Generative Adversarial Networks](https://mlsanity.com/paper/1812.04948/) (2018)
- [Photo-Realistic Single Image Super-Resolution Using a Generative Adversarial Network](https://mlsanity.com/paper/1609.04802/) (2016)
- [SwinIR: Image Restoration Using Swin Transformer](https://mlsanity.com/paper/2108.10257/) (2021)
- [Image Super-Resolution Using Very Deep Residual Channel Attention Networks](https://mlsanity.com/paper/1807.02758/) (2018)
- [FlowNet: Learning Optical Flow with Convolutional Networks](https://mlsanity.com/paper/1504.06852/) (2015)
- [Simple Baselines for Image Restoration](https://mlsanity.com/paper/2204.04676/) (2022)
- [Learning to See in the Dark](https://mlsanity.com/paper/1805.01934/) (2018)
- [Learning from Synthetic Humans](https://mlsanity.com/paper/1701.01370/) (2017)
- [BasicVSR++: Improving Video Super-Resolution with Enhanced Propagation and Alignment](https://mlsanity.com/paper/2104.13371/) (2021)
- [Virtual Worlds as Proxy for Multi-Object Tracking Analysis](https://mlsanity.com/paper/1605.06457/) (2016)
- [Training Deep Networks with Synthetic Data: Bridging the Reality Gap by Domain Randomization](https://mlsanity.com/paper/1804.06516/) (2018)
- [Learning from Simulated and Unsupervised Images through Adversarial Training](https://mlsanity.com/paper/1612.07828/) (2016)
- [Unprocessing Images for Learned Raw Denoising](https://mlsanity.com/paper/1811.11127/) (2018)
- [VRT: A Video Restoration Transformer](https://mlsanity.com/paper/2201.12288/) (2022)
- [Benchmarking Denoising Algorithms with Real Photographs](https://mlsanity.com/paper/1707.01313/) (2017)
- [Recurrent Video Restoration Transformer with Guided Deformable Attention](https://mlsanity.com/paper/2206.02146/) (2022)
- [A Physics-based Noise Formation Model for Extreme Low-light Raw Denoising](https://mlsanity.com/paper/2003.12751/) (2020)
- [Zooming Slow-Mo: Fast and Accurate One-Stage Space-Time Video Super-Resolution](https://mlsanity.com/paper/2002.11616/) (2020)
- [Supervised Raw Video Denoising with a Benchmark Dataset on Dynamic Scenes](https://mlsanity.com/paper/2003.14013/) (2020)
- [Practical Deep Raw Image Denoising on Mobile Devices](https://mlsanity.com/paper/2010.06935/) (2020)
- [A Poisson-Gaussian Denoising Dataset with Real Fluorescence Microscopy Images](https://mlsanity.com/paper/1812.10366/) (2018)
- [On Pre-Trained Image Features and Synthetic Images for Deep Learning](https://mlsanity.com/paper/1710.10710/) (2017)
- [Multi-Sensor Data Fusion for Cloud Removal in Global and All-Season Sentinel-2 Imagery](https://mlsanity.com/paper/2009.07683/) (2020)
- [Photorealistic Facial Texture Inference Using Deep Neural Networks](https://mlsanity.com/paper/1612.00523/) (2016)
- [Feature Mapping for Learning Fast and Accurate 3D Pose Inference from Synthetic Images](https://mlsanity.com/paper/1712.03904/) (2017)
- [Procedural Modeling and Physically Based Rendering for Synthetic Data Generation in Automotive Applications](https://mlsanity.com/paper/1710.06270/) (2017)
- [Learning from Synthetic Data: Addressing Domain Shift for Semantic Segmentation](https://mlsanity.com/paper/1711.06969/) (2017)

## Abstract (arXiv)

For low-level computer vision and image processing ML tasks, training on large datasets is critical for generalization. However, the standard practice of relying on real-world images primarily from the Internet comes with image quality, scalability, and privacy issues, especially in commercial contexts. To address this, we have developed a procedural synthetic data generation pipeline and dataset tailored to low-level vision tasks. Our Unreal engine-based synthetic data pipeline populates large scenes algorithmically with a combination of random 3D objects, materials, and geometric transformations. Then, we calibrate the camera noise profiles to synthesize the noisy images. From this pipeline, we generated a fully synthetic image denoising dataset (FSID) which consists of 175,000 noisy/clean image pairs. We then trained and validated a CNN-based denoising model, and demonstrated that the model trained on this synthetic data alone can achieve competitive denoising results when evaluated on real-world noisy images captured with smartphone cameras.