# A Physics-based Noise Formation Model for Extreme Low-light Raw Denoising

arXiv: https://arxiv.org/abs/2003.12751 · HTML: https://mlsanity.com/paper/2003.12751/

## Key facts

- Cited by 138 papers in the map (data through 7 Oct 2026).
- First posted on arXiv 28 Mar 2020.

## Built on by (most cited first, 30 of 138)

- [3D Common Corruptions and Data Augmentation](https://mlsanity.com/paper/2203.01441/) (2022)
- [You Only Need 90K Parameters to Adapt Light: A Light Weight Transformer for Image Enhancement and Exposure Correction](https://mlsanity.com/paper/2205.14871/) (2022)
- [Image Denoising: The Deep Learning Revolution and Beyond -- A Survey Paper --](https://mlsanity.com/paper/2301.03362/) (2023)
- [Physics-based Noise Modeling for Extreme Low-light Photography](https://mlsanity.com/paper/2108.02158/) (2021)
- [Rethinking Noise Synthesis and Modeling in Raw Denoising](https://mlsanity.com/paper/2110.04756/) (2021)
- [Masked Image Training for Generalizable Deep Image Denoising](https://mlsanity.com/paper/2303.13132/) (2023)
- [C2N: Practical Generative Noise Modeling for Real-World Denoising](https://mlsanity.com/paper/2202.09533/) (2022)
- [Adaptive Unfolding Total Variation Network for Low-Light Image Enhancement](https://mlsanity.com/paper/2110.00984/) (2021)
- [Realistic Blur Synthesis for Learning Image Deblurring](https://mlsanity.com/paper/2202.08771/) (2022)
- [Dancing under the stars: video denoising in starlight](https://mlsanity.com/paper/2204.04210/) (2022)
- [Instance Segmentation in the Dark](https://mlsanity.com/paper/2304.14298/) (2023)
- [IDR: Self-Supervised Image Denoising via Iterative Data Refinement](https://mlsanity.com/paper/2111.14358/) (2021)
- [Spatially Adaptive Self-Supervised Learning for Real-World Image Denoising](https://mlsanity.com/paper/2303.14934/) (2023)
- [ExposureDiffusion: Learning to Expose for Low-light Image Enhancement](https://mlsanity.com/paper/2307.07710/) (2023)
- [Unsupervised Image Denoising in Real-World Scenarios via Self-Collaboration Parallel Generative Adversarial Branches](https://mlsanity.com/paper/2308.06776/) (2023)
- [Modeling sRGB Camera Noise with Normalizing Flows](https://mlsanity.com/paper/2206.00812/) (2022)
- [Human Pose Estimation in Extremely Low-Light Conditions](https://mlsanity.com/paper/2303.15410/) (2023)
- [Towards Low Light Enhancement with RAW Images](https://mlsanity.com/paper/2112.14022/) (2021)
- [Variational Deep Image Restoration](https://mlsanity.com/paper/2207.01074/) (2022)
- [Day-to-Night Image Synthesis for Training Nighttime Neural ISPs](https://mlsanity.com/paper/2206.02715/) (2022)
- [Learnability Enhancement for Low-light Raw Denoising: Where Paired Real Data Meets Noise Modeling](https://mlsanity.com/paper/2207.06103/) (2022)
- [Diffusion in the Dark: A Diffusion Model for Low-Light Text Recognition](https://mlsanity.com/paper/2303.04291/) (2023)
- [DarkVisionNet: Low-Light Imaging via RGB-NIR Fusion with Deep Inconsistency Prior](https://mlsanity.com/paper/2303.06834/) (2023)
- [Towards General Low-Light Raw Noise Synthesis and Modeling](https://mlsanity.com/paper/2307.16508/) (2023)
- [RAW-Adapter: Adapting Pre-trained Visual Model to Camera RAW Images](https://mlsanity.com/paper/2408.14802/) (2024)
- [Rawgment: Noise-Accounted RAW Augmentation Enables Recognition in a Wide Variety of Environments](https://mlsanity.com/paper/2210.16046/) (2022)
- [Raw Image Reconstruction with Learned Compact Metadata](https://mlsanity.com/paper/2302.12995/) (2023)
- [Masked Video and Body-worn IMU Autoencoder for Egocentric Action Recognition](https://mlsanity.com/paper/2407.06628/) (2024)
- [Low-Light Image and Video Enhancement: A Comprehensive Survey and Beyond](https://mlsanity.com/paper/2212.10772/) (2022)
- [Binarized Low-light Raw Video Enhancement](https://mlsanity.com/paper/2403.19944/) (2024)

## Abstract (arXiv)

Lacking rich and realistic data, learned single image denoising algorithms generalize poorly to real raw images that do not resemble the data used for training. Although the problem can be alleviated by the heteroscedastic Gaussian model for noise synthesis, the noise sources caused by digital camera electronics are still largely overlooked, despite their significant effect on raw measurement, especially under extremely low-light condition. To address this issue, we present a highly accurate noise formation model based on the characteristics of CMOS photosensors, thereby enabling us to synthesize realistic samples that better match the physics of image formation process. Given the proposed noise model, we additionally propose a method to calibrate the noise parameters for available modern digital cameras, which is simple and reproducible for any new device. We systematically study the generalizability of a neural network trained with existing schemes, by introducing a new low-light denoising dataset that covers many modern digital cameras from diverse brands. Extensive empirical results collectively show that by utilizing our proposed noise formation model, a network can reach the capability as if it had been trained with rich real data, which demonstrates the effectiveness of our noise formation model.