# Self-supervision versus synthetic datasets: which is the lesser evil in the context of video denoising?

arXiv: https://arxiv.org/abs/2204.11493 · HTML: https://mlsanity.com/paper/2204.11493/

## Key facts

- Cited by 3 papers in the map (data through 14 Oct 2026).
- Builds on 30 of its 55 references that are in the map.
- First posted on arXiv 25 Apr 2022.

## Builds on (most cited first, 30 of 30)

- [Learning to See in the Dark](https://mlsanity.com/paper/1805.01934/) (2018)
- [Noise2Void - Learning Denoising from Single Noisy Images](https://mlsanity.com/paper/1811.10980/) (2018)
- [Noise2Self: Blind Denoising by Self-Supervision](https://mlsanity.com/paper/1901.11365/) (2019)
- [Toward Convolutional Blind Denoising of Real Photographs](https://mlsanity.com/paper/1807.04686/) (2018)
- [Unprocessing Images for Learned Raw Denoising](https://mlsanity.com/paper/1811.11127/) (2018)
- [CycleISP: Real Image Restoration via Improved Data Synthesis](https://mlsanity.com/paper/2003.07761/) (2020)
- [Benchmarking Denoising Algorithms with Real Photographs](https://mlsanity.com/paper/1707.01313/) (2017)
- [High-Quality Self-Supervised Deep Image Denoising](https://mlsanity.com/paper/1901.10277/) (2019)
- [FastDVDnet: Towards Real-Time Deep Video Denoising Without Flow Estimation](https://mlsanity.com/paper/1907.01361/) (2019)
- [Noisier2Noise: Learning to Denoise from Unpaired Noisy Data](https://mlsanity.com/paper/1910.11908/) (2019)
- [Noise Flow: Noise Modeling with Conditional Normalizing Flows](https://mlsanity.com/paper/1908.08453/) (2019)
- [Dual Adversarial Network: Toward Real-world Noise Removal and Noise Generation](https://mlsanity.com/paper/2007.05946/) (2020)
- [Neural Nearest Neighbors Networks](https://mlsanity.com/paper/1810.12575/) (2018)
- [Robust and interpretable blind image denoising via bias-free convolutional neural networks](https://mlsanity.com/paper/1906.05478/) (2019)
- [A deep convolutional neural network using directional wavelets for low-dose X-ray CT reconstruction](https://mlsanity.com/paper/1610.09736/) (2016)
- [Supervised Raw Video Denoising with a Benchmark Dataset on Dynamic Scenes](https://mlsanity.com/paper/2003.14013/) (2020)
- [Practical Deep Raw Image Denoising on Mobile Devices](https://mlsanity.com/paper/2010.06935/) (2020)
- [Physics-based Noise Modeling for Extreme Low-light Photography](https://mlsanity.com/paper/2108.02158/) (2021)
- [Probabilistic Noise2Void: Unsupervised Content-Aware Denoising](https://mlsanity.com/paper/1906.00651/) (2019)
- [Unsupervised Deep Video Denoising](https://mlsanity.com/paper/2011.15045/) (2020)
- [ViDeNN: Deep Blind Video Denoising](https://mlsanity.com/paper/1904.10898/) (2019)
- [Model-blind Video Denoising Via Frame-to-frame Training](https://mlsanity.com/paper/1811.12766/) (2018)
- [GRDN:Grouped Residual Dense Network for Real Image Denoising and GAN-based Real-world Noise Modeling](https://mlsanity.com/paper/1905.11172/) (2019)
- [Learning Camera-Aware Noise Models](https://mlsanity.com/paper/2008.09370/) (2020)
- [DeFlow: Learning Complex Image Degradations from Unpaired Data with Conditional Flows](https://mlsanity.com/paper/2101.05796/) (2021)
- [Self-Supervised training for blind multi-frame video denoising](https://mlsanity.com/paper/2004.06957/) (2020)
- [SAR Image Despeckling Using a Convolutional Neural Network](https://mlsanity.com/paper/1706.00552/) (2017)
- [Generating Training Data for Denoising Real RGB Images via Camera Pipeline Simulation](https://mlsanity.com/paper/1904.08825/) (2019)
- [Joint Demosaicking and Denoising by Fine-Tuning of Bursts of Raw Images](https://mlsanity.com/paper/1905.05092/) (2019)
- [Exploiting multi-temporal information for improved speckle reduction of Sentinel-1 SAR images by deep learning](https://mlsanity.com/paper/2102.00682/) (2021)

## Built on by (most cited first, 3 of 3)

- [Learning a Practical SDR-to-HDRTV Up-conversion using New Dataset and Degradation Models](https://mlsanity.com/paper/2303.13031/) (2023)
- [Learning Task-Oriented Flows to Mutually Guide Feature Alignment in Synthesized and Real Video Denoising](https://mlsanity.com/paper/2208.11803/) (2022)
- [Combining Pre- and Post-Demosaicking Noise Removal for RAW Video](https://mlsanity.com/paper/2410.02572/) (2024)

## Abstract (arXiv)

Supervised training has led to state-of-the-art results in image and video denoising. However, its application to real data is limited since it requires large datasets of noisy-clean pairs that are difficult to obtain. For this reason, networks are often trained on realistic synthetic data. More recently, some self-supervised frameworks have been proposed for training such denoising networks directly on the noisy data without requiring ground truth. On synthetic denoising problems supervised training outperforms self-supervised approaches, however in recent years the gap has become narrower, especially for video. In this paper, we propose a study aiming to determine which is the best approach to train denoising networks for real raw videos: supervision on synthetic realistic data or self-supervision on real data. A complete study with quantitative results in case of natural videos with real motion is impossible since no dataset with clean-noisy pairs exists. We address this issue by considering three independent experiments in which we compare the two frameworks. We found that self-supervision on the real data outperforms supervision on synthetic data, and that in normal illumination conditions the drop in performance is due to the synthetic ground truth generation, not the noise model.