# Dancing under the stars: video denoising in starlight

arXiv: https://arxiv.org/abs/2204.04210 · HTML: https://mlsanity.com/paper/2204.04210/

## Key facts

- Cited by 47 papers in the map (data through 14 Oct 2026).
- Builds on 30 of its 60 references that are in the map.
- First posted on arXiv 8 Apr 2022.

## Builds on (most cited first, 30 of 30)

- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [U-Net: Convolutional Networks for Biomedical Image Segmentation](https://mlsanity.com/paper/1505.04597/) (2015)
- [The Unreasonable Effectiveness of Deep Features as a Perceptual Metric](https://mlsanity.com/paper/1801.03924/) (2018)
- [Generative Adversarial Networks](https://mlsanity.com/paper/2203.00667/) (2022)
- [Improved Training of Wasserstein GANs](https://mlsanity.com/paper/1704.00028/) (2017)
- [Beyond a Gaussian Denoiser: Residual Learning of Deep CNN for Image Denoising](https://mlsanity.com/paper/1608.03981/) (2016)
- [Deep Image Prior](https://mlsanity.com/paper/1711.10925/) (2017)
- [Noise2Noise: Learning Image Restoration without Clean Data](https://mlsanity.com/paper/1803.04189/) (2018)
- [Learning to See in the Dark](https://mlsanity.com/paper/1805.01934/) (2018)
- [Noise2Void - Learning Denoising from Single Noisy Images](https://mlsanity.com/paper/1811.10980/) (2018)
- [Noise2Self: Blind Denoising by Self-Supervision](https://mlsanity.com/paper/1901.11365/) (2019)
- [Unprocessing Images for Learned Raw Denoising](https://mlsanity.com/paper/1811.11127/) (2018)
- [Benchmarking Denoising Algorithms with Real Photographs](https://mlsanity.com/paper/1707.01313/) (2017)
- [Burst Denoising with Kernel Prediction Networks](https://mlsanity.com/paper/1712.02327/) (2017)
- [A Physics-based Noise Formation Model for Extreme Low-light Raw Denoising](https://mlsanity.com/paper/2003.12751/) (2020)
- [MOTChallenge 2015: Towards a Benchmark for Multi-Target Tracking](https://mlsanity.com/paper/1504.01942/) (2015)
- [FastDVDnet: Towards Real-Time Deep Video Denoising Without Flow Estimation](https://mlsanity.com/paper/1907.01361/) (2019)
- [Noise Flow: Noise Modeling with Conditional Normalizing Flows](https://mlsanity.com/paper/1908.08453/) (2019)
- [Rethinking Noise Synthesis and Modeling in Raw Denoising](https://mlsanity.com/paper/2110.04756/) (2021)
- [Handheld Mobile Photography in Very Low Light](https://mlsanity.com/paper/1910.11336/) (2019)
- [Probabilistic Noise2Void: Unsupervised Content-Aware Denoising](https://mlsanity.com/paper/1906.00651/) (2019)
- [Deep Burst Denoising](https://mlsanity.com/paper/1712.05790/) (2017)
- [ViDeNN: Deep Blind Video Denoising](https://mlsanity.com/paper/1904.10898/) (2019)
- [Patch Craft: Video Denoising by Deep Modeling and Patch Matching](https://mlsanity.com/paper/2103.13767/) (2021)
- [RENOIR - A Dataset for Real Low-Light Image Noise Reduction](https://mlsanity.com/paper/1409.8230/) (2014)
- [Model-blind Video Denoising Via Frame-to-frame Training](https://mlsanity.com/paper/1811.12766/) (2018)
- [Learning Camera-Aware Noise Models](https://mlsanity.com/paper/2008.09370/) (2020)
- [High-level numerical simulations of noise in CCD and CMOS photosensors: review and tutorial](https://mlsanity.com/paper/1412.4031/) (2014)
- [Fully Unsupervised Probabilistic Noise2Void](https://mlsanity.com/paper/1911.12291/) (2019)
- [Class-Aware Fully-Convolutional Gaussian and Poisson Denoising](https://mlsanity.com/paper/1808.06562/) (2018)

## Built on by (most cited first, 30 of 47)

- [Physics-Informed Computer Vision: A Review and Perspectives](https://mlsanity.com/paper/2305.18035/) (2023)
- [DynamicISP: Dynamically Controlled Image Signal Processor for Image Recognition](https://mlsanity.com/paper/2211.01146/) (2022)
- [Diffusion in the Dark: A Diffusion Model for Low-Light Text Recognition](https://mlsanity.com/paper/2303.04291/) (2023)
- [Towards General Low-Light Raw Noise Synthesis and Modeling](https://mlsanity.com/paper/2307.16508/) (2023)
- [RAW-Adapter: Adapting Pre-trained Visual Model to Camera RAW Images](https://mlsanity.com/paper/2408.14802/) (2024)
- [Rawgment: Noise-Accounted RAW Augmentation Enables Recognition in a Wide Variety of Environments](https://mlsanity.com/paper/2210.16046/) (2022)
- [Low-Light Image and Video Enhancement: A Comprehensive Survey and Beyond](https://mlsanity.com/paper/2212.10772/) (2022)
- [AIM 2025 Low-light RAW Video Denoising Challenge: Dataset, Methods and Results](https://mlsanity.com/paper/2508.16830/) (2025)
- [DualDn: Dual-domain Denoising via Differentiable ISP](https://mlsanity.com/paper/2409.18783/) (2024)
- [Learning Physics-Informed Noise Models from Dark Frames for Low-Light Raw Image Denoising](https://mlsanity.com/paper/2310.09126/) (2023)
- [Shakes on a Plane: Unsupervised Depth Estimation from Unstabilized Photography](https://mlsanity.com/paper/2212.12324/) (2022)
- [SVNR: Spatially-variant Noise Removal with Denoising Diffusion](https://mlsanity.com/paper/2306.16052/) (2023)
- [BVI-RLV: A Fully Registered Dataset for Low-Light Video Enhancement](https://mlsanity.com/paper/2407.03535/) (2024)
- [Denoising the Deep Sky: Physics-Based CCD Noise Formation for Astronomical Imaging](https://mlsanity.com/paper/2601.23276/) (2026)
- [Modular Degradation Simulation and Restoration for Under-Display Camera](https://mlsanity.com/paper/2209.11455/) (2022)
- [Dual-Camera Joint Deblurring-Denoising](https://mlsanity.com/paper/2309.08826/) (2023)
- [Recurrent Spike-based Image Restoration under General Illumination](https://mlsanity.com/paper/2308.03018/) (2023)
- [DiffuseRAW: End-to-End Generative RAW Image Processing for Low-Light Images](https://mlsanity.com/paper/2402.18575/) (2023)
- [SeNM-VAE: Semi-Supervised Noise Modeling with Hierarchical Variational Autoencoder](https://mlsanity.com/paper/2403.17502/) (2024)
- [Dark Noise Diffusion: Noise Synthesis for Low-Light Image Denoising](https://mlsanity.com/paper/2503.11262/) (2025)
- [A Generative Model for Digital Camera Noise Synthesis](https://mlsanity.com/paper/2303.09199/) (2023)
- [Learned, uncertainty-driven adaptive acquisition for photon-efficient scanning microscopy](https://mlsanity.com/paper/2310.16102/) (2023)
- [Event Cameras Meet SPADs for High-Speed, Low-Bandwidth Imaging](https://mlsanity.com/paper/2404.11511/) (2024)
- [Quanta Video Restoration](https://mlsanity.com/paper/2410.14994/) (2024)
- [Zero-Shot Low Light Image Enhancement with Diffusion Prior](https://mlsanity.com/paper/2412.13401/) (2024)
- [Towards a General-Purpose Zero-Shot Synthetic Low-Light Image and Video Pipeline](https://mlsanity.com/paper/2504.12169/) (2025)
- [Contrastive Learning for Low-light Raw Denoising](https://mlsanity.com/paper/2305.03352/) (2023)
- [Make Explicit Calibration Implicit: Calibrate Denoiser Instead of the Noise Model](https://mlsanity.com/paper/2308.03448/) (2023)
- [Digitally Prototype Your Eye Tracker: Simulating Hardware Performance using 3D Synthetic Data](https://mlsanity.com/paper/2503.16742/) (2025)
- [RAW-Adapter: Adapting Pre-trained Visual Model to Camera RAW Images and A Benchmark](https://mlsanity.com/paper/2503.17027/) (2025)

## Abstract (arXiv)

Imaging in low light is extremely challenging due to low photon counts. Using sensitive CMOS cameras, it is currently possible to take videos at night under moonlight (0.05-0.3 lux illumination). In this paper, we demonstrate photorealistic video under starlight (no moon present, <0.001 lux) for the first time. To enable this, we develop a GAN-tuned physics-based noise model to more accurately represent camera noise at the lowest light levels. Using this noise model, we train a video denoiser using a combination of simulated noisy video clips and real noisy still images. We capture a 5-10 fps video dataset with significant motion at approximately 0.6-0.7 millilux with no active illumination. Comparing against alternative methods, we achieve improved video quality at the lowest light levels, demonstrating photorealistic video denoising in starlight for the first time.