# Efficient Multi-Stage Video Denoising with Recurrent Spatio-Temporal Fusion

arXiv: https://arxiv.org/abs/2103.05407 · HTML: https://mlsanity.com/paper/2103.05407/

## Key facts

- Cited by 36 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 9 Mar 2021.

## Built on by (most cited first, 30 of 36)

- [VRT: A Video Restoration Transformer](https://mlsanity.com/paper/2201.12288/) (2022)
- [Image Denoising: The Deep Learning Revolution and Beyond -- A Survey Paper --](https://mlsanity.com/paper/2301.03362/) (2023)
- [A Simple Baseline for Video Restoration with Grouped Spatial-temporal Shift](https://mlsanity.com/paper/2206.10810/) (2022)
- [Unidirectional Video Denoising by Mimicking Backward Recurrent Modules with Look-ahead Forward Ones](https://mlsanity.com/paper/2204.05532/) (2022)
- [Neural Compression-Based Feature Learning for Video Restoration](https://mlsanity.com/paper/2203.09208/) (2022)
- [Binarized Low-light Raw Video Enhancement](https://mlsanity.com/paper/2403.19944/) (2024)
- [Efficient Burst Raw Denoising with Variance Stabilization and Multi-frequency Denoising Network](https://mlsanity.com/paper/2205.04721/) (2022)
- [Real-time Streaming Video Denoising with Bidirectional Buffers](https://mlsanity.com/paper/2207.06937/) (2022)
- [NTIRE 2022 Challenge on High Dynamic Range Imaging: Methods and Results](https://mlsanity.com/paper/2205.12633/) (2022)
- [NTIRE 2024 Restore Any Image Model (RAIM) in the Wild Challenge](https://mlsanity.com/paper/2405.09923/) (2024)
- [RViDeformer: Efficient Raw Video Denoising Transformer with a Larger Benchmark Dataset](https://mlsanity.com/paper/2305.00767/) (2023)
- [Recurrent Self-Supervised Video Denoising with Denser Receptive Field](https://mlsanity.com/paper/2308.03608/) (2023)
- [Temporal As a Plugin: Unsupervised Video Denoising with Pre-Trained Image Denoisers](https://mlsanity.com/paper/2409.11256/) (2024)
- [Learning Task-Oriented Flows to Mutually Guide Feature Alignment in Synthesized and Real Video Denoising](https://mlsanity.com/paper/2208.11803/) (2022)
- [Recurrent Spike-based Image Restoration under General Illumination](https://mlsanity.com/paper/2308.03018/) (2023)
- [Towards Blind Bitstream-corrupted Video Recovery via a Visual Foundation Model-driven Framework](https://mlsanity.com/paper/2507.22481/) (2025)
- [Priors in Deep Image Restoration and Enhancement: A Survey](https://mlsanity.com/paper/2206.02070/) (2022)
- [Learning Physical-Spatio-Temporal Features for Video Shadow Removal](https://mlsanity.com/paper/2303.09370/) (2023)
- [Reference-based Restoration of Digitized Analog Videotapes](https://mlsanity.com/paper/2310.14926/) (2023)
- [Quanta Video Restoration](https://mlsanity.com/paper/2410.14994/) (2024)
- [Online Video Quality Enhancement with Spatial-Temporal Look-up Tables](https://mlsanity.com/paper/2311.13616/) (2023)
- [Classic Video Denoising in a Machine Learning World: Robust, Fast, and Controllable](https://mlsanity.com/paper/2504.03136/) (2025)
- [Joint Flow And Feature Refinement Using Attention For Video Restoration](https://mlsanity.com/paper/2505.16434/) (2025)
- [Towards Real-World Video Denosing: A Practical Video Denosing Dataset and Network](https://mlsanity.com/paper/2207.01356/) (2022)
- [Gated Recurrent Unit for Video Denoising](https://mlsanity.com/paper/2210.09135/) (2022)
- [Improving Inference Performance of Machine Learning with the Divide-and-Conquer Principle](https://mlsanity.com/paper/2301.05099/) (2023)
- [Technique Report of CVPR 2024 PBDL Challenges](https://mlsanity.com/paper/2406.10744/) (2024)
- [Adapting MIMO video restoration networks to low latency constraints](https://mlsanity.com/paper/2408.12439/) (2024)
- [A Practical Gated Recurrent Transformer Network Incorporating Multiple Fusions for Video Denoising](https://mlsanity.com/paper/2409.06603/) (2024)
- [Combining Pre- and Post-Demosaicking Noise Removal for RAW Video](https://mlsanity.com/paper/2410.02572/) (2024)

## Abstract (arXiv)

In recent years, denoising methods based on deep learning have achieved unparalleled performance at the cost of large computational complexity. In this work, we propose an Efficient Multi-stage Video Denoising algorithm, called EMVD, to drastically reduce the complexity while maintaining or even improving the performance. First, a fusion stage reduces the noise through a recursive combination of all past frames in the video. Then, a denoising stage removes the noise in the fused frame. Finally, a refinement stage restores the missing high frequency in the denoised frame. All stages operate on a transform-domain representation obtained by learnable and invertible linear operators which simultaneously increase accuracy and decrease complexity of the model. A single loss on the final output is sufficient for successful convergence, hence making EMVD easy to train. Experiments on real raw data demonstrate that EMVD outperforms the state of the art when complexity is constrained, and even remains competitive against methods whose complexities are several orders of magnitude higher. Further, the low complexity and memory requirements of EMVD enable real-time video denoising on commercial SoC in mobile devices.