# Combining Pre- and Post-Demosaicking Noise Removal for RAW Video

arXiv: https://arxiv.org/abs/2410.02572 · HTML: https://mlsanity.com/paper/2410.02572/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 43 of its 108 references that are in the map.
- First posted on arXiv 3 Oct 2024.

## Builds on (most cited first, 30 of 43)

- [Restormer: Efficient Transformer for High-Resolution Image Restoration](https://mlsanity.com/paper/2111.09881/) (2021)
- [Deformable Convolutional Networks](https://mlsanity.com/paper/1703.06211/) (2017)
- [Beyond a Gaussian Denoiser: Residual Learning of Deep CNN for Image Denoising](https://mlsanity.com/paper/1608.03981/) (2016)
- [Simple Baselines for Image Restoration](https://mlsanity.com/paper/2204.04676/) (2022)
- [Video Enhancement with Task-Oriented Flow](https://mlsanity.com/paper/1711.09078/) (2017)
- [Noise2Noise: Learning Image Restoration without Clean Data](https://mlsanity.com/paper/1803.04189/) (2018)
- [FFDNet: Toward a Fast and Flexible Solution for CNN based Image Denoising](https://mlsanity.com/paper/1710.04026/) (2017)
- [EDVR: Video Restoration with Enhanced Deformable Convolutional Networks](https://mlsanity.com/paper/1905.02716/) (2019)
- [Learning to See in the Dark](https://mlsanity.com/paper/1805.01934/) (2018)
- [Noise2Void - Learning Denoising from Single Noisy Images](https://mlsanity.com/paper/1811.10980/) (2018)
- [Noise2Self: Blind Denoising by Self-Supervision](https://mlsanity.com/paper/1901.11365/) (2019)
- [Toward Convolutional Blind Denoising of Real Photographs](https://mlsanity.com/paper/1807.04686/) (2018)
- [Unprocessing Images for Learned Raw Denoising](https://mlsanity.com/paper/1811.11127/) (2018)
- [VRT: A Video Restoration Transformer](https://mlsanity.com/paper/2201.12288/) (2022)
- [CycleISP: Real Image Restoration via Improved Data Synthesis](https://mlsanity.com/paper/2003.07761/) (2020)
- [Real Image Denoising with Feature Attention](https://mlsanity.com/paper/1904.07396/) (2019)
- [Recurrent Video Restoration Transformer with Guided Deformable Attention](https://mlsanity.com/paper/2206.02146/) (2022)
- [High-Quality Self-Supervised Deep Image Denoising](https://mlsanity.com/paper/1901.10277/) (2019)
- [A Physics-based Noise Formation Model for Extreme Low-light Raw Denoising](https://mlsanity.com/paper/2003.12751/) (2020)
- [FastDVDnet: Towards Real-Time Deep Video Denoising Without Flow Estimation](https://mlsanity.com/paper/1907.01361/) (2019)
- [Supervised Raw Video Denoising with a Benchmark Dataset on Dynamic Scenes](https://mlsanity.com/paper/2003.14013/) (2020)
- [Practical Deep Raw Image Denoising on Mobile Devices](https://mlsanity.com/paper/2010.06935/) (2020)
- [DVDnet: A Fast Network for Deep Video Denoising](https://mlsanity.com/paper/1906.11890/) (2019)
- [Unsupervised Deep Video Denoising](https://mlsanity.com/paper/2011.15045/) (2020)
- [Patch Craft: Video Denoising by Deep Modeling and Patch Matching](https://mlsanity.com/paper/2103.13767/) (2021)
- [Efficient Multi-Stage Video Denoising with Recurrent Spatio-Temporal Fusion](https://mlsanity.com/paper/2103.05407/) (2021)
- [Model-blind Video Denoising Via Frame-to-frame Training](https://mlsanity.com/paper/1811.12766/) (2018)
- [Learning Raw Image Denoising with Bayer Pattern Unification and Bayer Preserving Augmentation](https://mlsanity.com/paper/1904.12945/) (2019)
- [Iterative Joint Image Demosaicking and Denoising using a Residual Denoising Network](https://mlsanity.com/paper/1807.06403/) (2018)
- [Self-Supervised training for blind multi-frame video denoising](https://mlsanity.com/paper/2004.06957/) (2020)

## Abstract (arXiv)

Denoising is one of the fundamental steps of the processing pipeline that converts data captured by a camera sensor into a display-ready image or video. It is generally performed early in the pipeline, usually before demosaicking, although studies swapping their order or even conducting them jointly have been proposed. With the advent of deep learning, the quality of denoising algorithms has steadily increased. Even so, modern neural networks still have a hard time adapting to new noise levels and scenes, which is indispensable for real-world applications. With those in mind, we propose a self-similarity-based denoising scheme that weights both a pre- and a post-demosaicking denoiser for Bayer-patterned CFA video data. We show that a balance between the two leads to better image quality, and we empirically find that higher noise levels benefit from a higher influence pre-demosaicking. We also integrate temporal trajectory prefiltering steps before each denoiser, which further improve texture reconstruction. The proposed method only requires an estimation of the noise model at the sensor, accurately adapts to any noise level, and is competitive with the state of the art, making it suitable for real-world videography.