# Real-time Streaming Video Denoising with Bidirectional Buffers

arXiv: https://arxiv.org/abs/2207.06937 · HTML: https://mlsanity.com/paper/2207.06937/

## Key facts

- Cited by 13 papers in the map (data through 14 Oct 2026).
- Builds on 25 of its 35 references that are in the map.
- First posted on arXiv 14 Jul 2022.

## Builds on (most cited first, 25 of 25)

- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [U-Net: Convolutional Networks for Biomedical Image Segmentation](https://mlsanity.com/paper/1505.04597/) (2015)
- [SwinIR: Image Restoration Using Swin Transformer](https://mlsanity.com/paper/2108.10257/) (2021)
- [Restormer: Efficient Transformer for High-Resolution Image Restoration](https://mlsanity.com/paper/2111.09881/) (2021)
- [Enhanced Deep Residual Networks for Single Image Super-Resolution](https://mlsanity.com/paper/1707.02921/) (2017)
- [Beyond a Gaussian Denoiser: Residual Learning of Deep CNN for Image Denoising](https://mlsanity.com/paper/1608.03981/) (2016)
- [ESRGAN: Enhanced Super-Resolution Generative Adversarial Networks](https://mlsanity.com/paper/1809.00219/) (2018)
- [TSM: Temporal Shift Module for Efficient Video Understanding](https://mlsanity.com/paper/1811.08383/) (2018)
- [Deep Multi-scale Convolutional Neural Network for Dynamic Scene Deblurring](https://mlsanity.com/paper/1612.02177/) (2016)
- [MOT16: A Benchmark for Multi-Object Tracking](https://mlsanity.com/paper/1603.00831/) (2016)
- [EDVR: Video Restoration with Enhanced Deformable Convolutional Networks](https://mlsanity.com/paper/1905.02716/) (2019)
- [BasicVSR: The Search for Essential Components in Video Super-Resolution and Beyond](https://mlsanity.com/paper/2012.02181/) (2020)
- [Burst Denoising with Kernel Prediction Networks](https://mlsanity.com/paper/1712.02327/) (2017)
- [MoViNets: Mobile Video Networks for Efficient Video Recognition](https://mlsanity.com/paper/2103.11511/) (2021)
- [FastDVDnet: Towards Real-Time Deep Video Denoising Without Flow Estimation](https://mlsanity.com/paper/1907.01361/) (2019)
- [Supervised Raw Video Denoising with a Benchmark Dataset on Dynamic Scenes](https://mlsanity.com/paper/2003.14013/) (2020)
- [DVDnet: A Fast Network for Deep Video Denoising](https://mlsanity.com/paper/1906.11890/) (2019)
- [W-Net: A Deep Model for Fully Unsupervised Image Segmentation](https://mlsanity.com/paper/1711.08506/) (2017)
- [Basis Prediction Networks for Effective Burst Denoising with Large Kernels](https://mlsanity.com/paper/1912.04421/) (2019)
- [Unsupervised Deep Video Denoising](https://mlsanity.com/paper/2011.15045/) (2020)
- [Patch Craft: Video Denoising by Deep Modeling and Patch Matching](https://mlsanity.com/paper/2103.13767/) (2021)
- [Efficient Multi-Stage Video Denoising with Recurrent Spatio-Temporal Fusion](https://mlsanity.com/paper/2103.05407/) (2021)
- [VIDOSAT: High-dimensional Sparsifying Transform Learning for Online Video Denoising](https://mlsanity.com/paper/1710.00947/) (2017)
- [Non-Local Video Denoising by CNN](https://mlsanity.com/paper/1811.12758/) (2018)
- [Efficient Deep Image Denoising via Class Specific Convolution](https://mlsanity.com/paper/2103.01624/) (2021)

## Built on by (most cited first, 13 of 13)

- [DiffSHEG: A Diffusion-Based Approach for Real-Time Speech-driven Holistic 3D Expression and Gesture Generation](https://mlsanity.com/paper/2401.04747/) (2024)
- [Learning Truncated Causal History Model for Video Restoration](https://mlsanity.com/paper/2410.03936/) (2024)
- [RViDeformer: Efficient Raw Video Denoising Transformer with a Larger Benchmark Dataset](https://mlsanity.com/paper/2305.00767/) (2023)
- [Recurrent Self-Supervised Video Denoising with Denser Receptive Field](https://mlsanity.com/paper/2308.03608/) (2023)
- [Unsupervised HDR Image and Video Tone Mapping via Contrastive Learning](https://mlsanity.com/paper/2303.07327/) (2023)
- [Deep Variation Prior: Joint Image Denoising and Noise Variance Estimation without Clean Data](https://mlsanity.com/paper/2209.09214/) (2022)
- [VJT: A Video Transformer on Joint Tasks of Deblurring, Low-light Enhancement and Denoising](https://mlsanity.com/paper/2401.14754/) (2024)
- [Adapting MIMO video restoration networks to low latency constraints](https://mlsanity.com/paper/2408.12439/) (2024)
- [Combining Pre- and Post-Demosaicking Noise Removal for RAW Video](https://mlsanity.com/paper/2410.02572/) (2024)
- [Video Denoising in Fluorescence Guided Surgery](https://mlsanity.com/paper/2411.09798/) (2024)
- [AsynFusion: Towards Asynchronous Latent Consistency Models for Decoupled Whole-Body Audio-Driven Avatars](https://mlsanity.com/paper/2505.15058/) (2025)
- [UHD-GPGNet: UHD Video Denoising via Gaussian-Process-Guided Local Spatio-Temporal Modeling](https://mlsanity.com/paper/2604.11014/) (2026)
- [Multi-frame Restoration for 10 Hz Lissajous Confocal Laser Endomicroscopy](https://mlsanity.com/paper/2605.00527/) (2026)

## Abstract (arXiv)

Video streams are delivered continuously to save the cost of storage and device memory. Real-time denoising algorithms are typically adopted on the user device to remove the noise involved during the shooting and transmission of video streams. However, sliding-window-based methods feed multiple input frames for a single output and lack computation efficiency. Recent multi-output inference works propagate the bidirectional temporal feature with a parallel or recurrent framework, which either suffers from performance drops on the temporal edges of clips or can not achieve online inference. In this paper, we propose a Bidirectional Streaming Video Denoising (BSVD) framework, to achieve high-fidelity real-time denoising for streaming videos with both past and future temporal receptive fields. The bidirectional temporal fusion for online inference is considered not applicable in the MoViNet. However, we introduce a novel Bidirectional Buffer Block as the core module of our BSVD, which makes it possible during our pipeline-style inference. In addition, our method is concise and flexible to be utilized in both non-blind and blind video denoising. We compare our model with various state-of-the-art video denoising models qualitatively and quantitatively on synthetic and real noise. Our method outperforms previous methods in terms of restoration fidelity and runtime. Our source code is publicly available at https://github.com/ChenyangQiQi/BSVD