# Physics-based Noise Modeling for Extreme Low-light Photography

arXiv: https://arxiv.org/abs/2108.02158 · HTML: https://mlsanity.com/paper/2108.02158/

## Key facts

- Cited by 67 papers in the map (data through 14 Oct 2026).
- Builds on 26 of its 97 references that are in the map.
- First posted on arXiv 4 Aug 2021.

## Builds on (most cited first, 26 of 26)

- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [U-Net: Convolutional Networks for Biomedical Image Segmentation](https://mlsanity.com/paper/1505.04597/) (2015)
- [Very Deep Convolutional Networks for Large-Scale Image Recognition](https://mlsanity.com/paper/1409.1556/) (2014)
- [RAFT: Recurrent All-Pairs Field Transforms for Optical Flow](https://mlsanity.com/paper/2003.12039/) (2020)
- [Towards Robust Monocular Depth Estimation: Mixing Datasets for Zero-shot Cross-dataset Transfer](https://mlsanity.com/paper/1907.01341/) (2019)
- [Beyond a Gaussian Denoiser: Residual Learning of Deep CNN for Image Denoising](https://mlsanity.com/paper/1608.03981/) (2016)
- [Noise2Noise: Learning Image Restoration without Clean Data](https://mlsanity.com/paper/1803.04189/) (2018)
- [Learning to See in the Dark](https://mlsanity.com/paper/1805.01934/) (2018)
- [Noise2Void - Learning Denoising from Single Noisy Images](https://mlsanity.com/paper/1811.10980/) (2018)
- [Toward Convolutional Blind Denoising of Real Photographs](https://mlsanity.com/paper/1807.04686/) (2018)
- [Unprocessing Images for Learned Raw Denoising](https://mlsanity.com/paper/1811.11127/) (2018)
- [MemNet: A Persistent Memory Network for Image Restoration](https://mlsanity.com/paper/1708.02209/) (2017)
- [Benchmarking Denoising Algorithms with Real Photographs](https://mlsanity.com/paper/1707.01313/) (2017)
- [Real Image Denoising with Feature Attention](https://mlsanity.com/paper/1904.07396/) (2019)
- [Burst Denoising with Kernel Prediction Networks](https://mlsanity.com/paper/1712.02327/) (2017)
- [A Physics-based Noise Formation Model for Extreme Low-light Raw Denoising](https://mlsanity.com/paper/2003.12751/) (2020)
- [FastDVDnet: Towards Real-Time Deep Video Denoising Without Flow Estimation](https://mlsanity.com/paper/1907.01361/) (2019)
- [Noise Flow: Noise Modeling with Conditional Normalizing Flows](https://mlsanity.com/paper/1908.08453/) (2019)
- [Coded aperture compressive temporal imaging](https://mlsanity.com/paper/1302.2575/) (2013)
- [Handheld Mobile Photography in Very Low Light](https://mlsanity.com/paper/1910.11336/) (2019)
- [Basis Prediction Networks for Effective Burst Denoising with Large Kernels](https://mlsanity.com/paper/1912.04421/) (2019)
- [A Trilateral Weighted Sparse Coding Scheme for Real-World Image Denoising](https://mlsanity.com/paper/1807.04364/) (2018)
- [Multi-channel Weighted Nuclear Norm Minimization for Real Color Image Denoising](https://mlsanity.com/paper/1705.09912/) (2017)
- [Learning Camera-Aware Noise Models](https://mlsanity.com/paper/2008.09370/) (2020)
- [On learning optimized reaction diffusion processes for effective image restoration](https://mlsanity.com/paper/1503.05768/) (2015)
- [High-level numerical simulations of noise in CCD and CMOS photosensors: review and tutorial](https://mlsanity.com/paper/1412.4031/) (2014)

## Built on by (most cited first, 30 of 67)

- [Frequency-aware Feature Fusion for Dense Image Prediction](https://mlsanity.com/paper/2408.12879/) (2024)
- [Instance Segmentation in the Dark](https://mlsanity.com/paper/2304.14298/) (2023)
- [RawHDR: High Dynamic Range Image Reconstruction from a Single Raw Image](https://mlsanity.com/paper/2309.02020/) (2023)
- [Human Pose Estimation in Extremely Low-Light Conditions](https://mlsanity.com/paper/2303.15410/) (2023)
- [Lighting Every Darkness with 3DGS: Fast Training and Real-Time Rendering for HDR View Synthesis](https://mlsanity.com/paper/2406.06216/) (2024)
- [Day-to-Night Image Synthesis for Training Nighttime Neural ISPs](https://mlsanity.com/paper/2206.02715/) (2022)
- [Learnability Enhancement for Low-light Raw Denoising: Where Paired Real Data Meets Noise Modeling](https://mlsanity.com/paper/2207.06103/) (2022)
- [Diffusion in the Dark: A Diffusion Model for Low-Light Text Recognition](https://mlsanity.com/paper/2303.04291/) (2023)
- [Binarized Low-light Raw Video Enhancement](https://mlsanity.com/paper/2403.19944/) (2024)
- [Noise Calibration and Spatial-Frequency Interactive Network for STEM Image Enhancement](https://mlsanity.com/paper/2504.02555/) (2025)
- [Estimating Fine-Grained Noise Model via Contrastive Learning](https://mlsanity.com/paper/2204.01716/) (2022)
- [LRT: An Efficient Low-Light Restoration Transformer for Dark Light Field Images](https://mlsanity.com/paper/2209.02197/) (2022)
- [Noise Self-Regression: A New Learning Paradigm to Enhance Low-Light Images Without Task-Related Data](https://mlsanity.com/paper/2211.04700/) (2022)
- [MIPI 2024 Challenge on Few-shot RAW Image Denoising: Methods and Results](https://mlsanity.com/paper/2406.07006/) (2024)
- [DualDn: Dual-domain Denoising via Differentiable ISP](https://mlsanity.com/paper/2409.18783/) (2024)
- [Learning to See in the Extremely Dark](https://mlsanity.com/paper/2506.21132/) (2025)
- [Learning Physics-Informed Noise Models from Dark Frames for Low-Light Raw Image Denoising](https://mlsanity.com/paper/2310.09126/) (2023)
- [Source Matters: Source Dataset Impact on Model Robustness in Medical Imaging](https://mlsanity.com/paper/2403.04484/) (2024)
- [When Semantic Segmentation Meets Frequency Aliasing](https://mlsanity.com/paper/2403.09065/) (2024)
- [Flow-Anything: Learning Real-World Optical Flow Estimation from Large-Scale Single-view Images](https://mlsanity.com/paper/2506.07740/) (2025)
- [Self-Supervised Scalable Deep Compressed Sensing](https://mlsanity.com/paper/2308.13777/) (2023)
- [Denoising the Deep Sky: Physics-Based CCD Noise Formation for Astronomical Imaging](https://mlsanity.com/paper/2601.23276/) (2026)
- [Domain-Adaptive 2D Human Pose Estimation via Dual Teachers in Extremely Low-Light Conditions](https://mlsanity.com/paper/2407.15451/) (2024)
- [Retinex-RAWMamba: Bridging Demosaicing and Denoising for Low-Light RAW Image Enhancement](https://mlsanity.com/paper/2409.07040/) (2024)
- [Positive2Negative: Breaking the Information-Lossy Barrier in Self-Supervised Single Image Denoising](https://mlsanity.com/paper/2412.16460/) (2024)
- [Supervise-assisted Multi-modality Fusion Diffusion Model for PET Restoration](https://mlsanity.com/paper/2602.11545/) (2026)
- [Dark-ISP: Enhancing RAW Image Processing for Low-Light Object Detection](https://mlsanity.com/paper/2509.09183/) (2025)
- [Self-Supervised Monocular Depth Estimation in the Dark: Towards Data Distribution Compensation](https://mlsanity.com/paper/2404.13854/) (2024)
- [Dark Noise Diffusion: Noise Synthesis for Low-Light Image Denoising](https://mlsanity.com/paper/2503.11262/) (2025)
- [YOND: Practical Blind Raw Image Denoising Free from Camera-Specific Data Dependency](https://mlsanity.com/paper/2506.03645/) (2025)

## Abstract (arXiv)

Enhancing the visibility in extreme low-light environments is a challenging task. Under nearly lightless condition, existing image denoising methods could easily break down due to significantly low SNR. In this paper, we systematically study the noise statistics in the imaging pipeline of CMOS photosensors, and formulate a comprehensive noise model that can accurately characterize the real noise structures. Our novel model considers the noise sources caused by digital camera electronics which are largely overlooked by existing methods yet have significant influence on raw measurement in the dark. It provides a way to decouple the intricate noise structure into different statistical distributions with physical interpretations. Moreover, our noise model can be used to synthesize realistic training data for learning-based low-light denoising algorithms. In this regard, although promising results have been shown recently with deep convolutional neural networks, the success heavily depends on abundant noisy clean image pairs for training, which are tremendously difficult to obtain in practice. Generalizing their trained models to images from new devices is also problematic. Extensive experiments on multiple low-light denoising datasets -- including a newly collected one in this work covering various devices -- show that a deep neural network trained with our proposed noise formation model can reach surprisingly-high accuracy. The results are on par with or sometimes even outperform training with paired real data, opening a new door to real-world extreme low-light photography.