# Neural Compression-Based Feature Learning for Video Restoration

arXiv: https://arxiv.org/abs/2203.09208 · HTML: https://mlsanity.com/paper/2203.09208/

## Key facts

- Cited by 23 papers in the map (data through 14 Oct 2026).
- Builds on 35 of its 52 references that are in the map.
- First posted on arXiv 17 Mar 2022.

## Builds on (most cited first, 30 of 35)

- [U-Net: Convolutional Networks for Biomedical Image Segmentation](https://mlsanity.com/paper/1505.04597/) (2015)
- [Squeeze-and-Excitation Networks](https://mlsanity.com/paper/1709.01507/) (2017)
- [Enhanced Deep Residual Networks for Single Image Super-Resolution](https://mlsanity.com/paper/1707.02921/) (2017)
- [Image Super-Resolution Using Very Deep Residual Channel Attention Networks](https://mlsanity.com/paper/1807.02758/) (2018)
- [Variational image compression with a scale hyperprior](https://mlsanity.com/paper/1802.01436/) (2018)
- [Multi-Stage Progressive Image Restoration](https://mlsanity.com/paper/2102.02808/) (2021)
- [End-to-end Optimized Image Compression](https://mlsanity.com/paper/1611.01704/) (2016)
- [Joint Autoregressive and Hierarchical Priors for Learned Image Compression](https://mlsanity.com/paper/1809.02736/) (2018)
- [Video Enhancement with Task-Oriented Flow](https://mlsanity.com/paper/1711.09078/) (2017)
- [EDVR: Video Restoration with Enhanced Deformable Convolutional Networks](https://mlsanity.com/paper/1905.02716/) (2019)
- [Learning to See in the Dark](https://mlsanity.com/paper/1805.01934/) (2018)
- [Optical Flow Estimation using a Spatial Pyramid Network](https://mlsanity.com/paper/1611.00850/) (2016)
- [BasicVSR++: Improving Video Super-Resolution with Enhanced Propagation and Alignment](https://mlsanity.com/paper/2104.13371/) (2021)
- [DVC: An End-to-end Deep Video Compression Framework](https://mlsanity.com/paper/1812.00101/) (2018)
- [FFA-Net: Feature Fusion Attention Network for Single Image Dehazing](https://mlsanity.com/paper/1911.07559/) (2019)
- [BasicVSR: The Search for Essential Components in Video Super-Resolution and Beyond](https://mlsanity.com/paper/2012.02181/) (2020)
- [Single Image Super-Resolution via a Holistic Attention Network](https://mlsanity.com/paper/2008.08767/) (2020)
- [Residual Non-local Attention Networks for Image Restoration](https://mlsanity.com/paper/1903.10082/) (2019)
- [Multi-Scale Boosted Dehazing Network with Dense Feature Fusion](https://mlsanity.com/paper/2004.13388/) (2020)
- [GridDehazeNet: Attention-Based Multi-Scale Network for Image Dehazing](https://mlsanity.com/paper/1908.03245/) (2019)
- [Deep Contextual Video Compression](https://mlsanity.com/paper/2109.15047/) (2021)
- [FastDVDnet: Towards Real-Time Deep Video Denoising Without Flow Estimation](https://mlsanity.com/paper/1907.01361/) (2019)
- [Density Modeling of Images using a Generalized Normalization Transformation](https://mlsanity.com/paper/1511.06281/) (2015)
- [M-LVC: Multiple Frames Prediction for Learned Video Compression](https://mlsanity.com/paper/2004.10290/) (2020)
- [Supervised Raw Video Denoising with a Benchmark Dataset on Dynamic Scenes](https://mlsanity.com/paper/2003.14013/) (2020)
- [Dual Residual Networks Leveraging the Potential of Paired Operations for Image Restoration](https://mlsanity.com/paper/1903.08817/) (2019)
- [DVDnet: A Fast Network for Deep Video Denoising](https://mlsanity.com/paper/1906.11890/) (2019)
- [W-Net: A Deep Model for Fully Unsupervised Image Segmentation](https://mlsanity.com/paper/1711.08506/) (2017)
- [Robust Video Content Alignment and Compensation for Rain Removal in a CNN Framework](https://mlsanity.com/paper/1803.10433/) (2018)
- [Conditional Entropy Coding for Efficient Video Compression](https://mlsanity.com/paper/2008.09180/) (2020)

## Built on by (most cited first, 23 of 23)

- [Hybrid Spatial-Temporal Entropy Modelling for Neural Video Compression](https://mlsanity.com/paper/2207.05894/) (2022)
- [Neural Video Compression with Feature Modulation](https://mlsanity.com/paper/2402.17414/) (2024)
- [A Simple Baseline for Video Restoration with Grouped Spatial-temporal Shift](https://mlsanity.com/paper/2206.10810/) (2022)
- [RainMamba: Enhanced Locality Learning with State Space Models for Video Deraining](https://mlsanity.com/paper/2407.21773/) (2024)
- [Video Dehazing via a Multi-Range Temporal Alignment Network with Physical Prior](https://mlsanity.com/paper/2303.09757/) (2023)
- [On the Generalization of BasicVSR++ to Video Deblurring and Denoising](https://mlsanity.com/paper/2204.05308/) (2022)
- [VNVC: A Versatile Neural Video Coding Framework for Efficient Human-Machine Vision](https://mlsanity.com/paper/2306.10681/) (2023)
- [Learning Truncated Causal History Model for Video Restoration](https://mlsanity.com/paper/2410.03936/) (2024)
- [LAVIB: A Large-scale Video Interpolation Benchmark](https://mlsanity.com/paper/2406.09754/) (2024)
- [Recurrent Self-Supervised Video Denoising with Denser Receptive Field](https://mlsanity.com/paper/2308.03608/) (2023)
- [Driving-Video Dehazing with Non-Aligned Regularization for Safety Assistance](https://mlsanity.com/paper/2405.09996/) (2024)
- [Learning Task-Oriented Flows to Mutually Guide Feature Alignment in Synthesized and Real Video Denoising](https://mlsanity.com/paper/2208.11803/) (2022)
- [Cross-Consistent Deep Unfolding Network for Adaptive All-In-One Video Restoration](https://mlsanity.com/paper/2309.01627/) (2023)
- [Grounding Degradations in Natural Language for All-In-One Video Restoration](https://mlsanity.com/paper/2507.14851/) (2025)
- [PNeRV: A Polynomial Neural Representation for Videos](https://mlsanity.com/paper/2406.19299/) (2024)
- [Bitrate-Controlled Diffusion for Disentangling Motion and Content in Video](https://mlsanity.com/paper/2509.08376/) (2025)
- [Removing Multiple Hybrid Adverse Weather in Video via a Unified Model](https://mlsanity.com/paper/2503.06200/) (2025)
- [Unsupervised Methods for Video Quality Improvement: A Survey of Restoration and Enhancement Techniques](https://mlsanity.com/paper/2507.08375/) (2025)
- [ViStripformer: A Token-Efficient Transformer for Versatile Video Restoration](https://mlsanity.com/paper/2312.14502/) (2023)
- [Adapting MIMO video restoration networks to low latency constraints](https://mlsanity.com/paper/2408.12439/) (2024)
- [SCENE: Semantic-aware Codec Enhancement with Neural Embeddings](https://mlsanity.com/paper/2601.22189/) (2026)
- [PLAF: Pixel-wise Language-Aligned Feature Extraction for Efficient 3D Scene Understanding](https://mlsanity.com/paper/2604.15770/) (2026)
- [S3VD: Semantic-Guidance Spatio-Temporal Scanning for Video Deraining](https://mlsanity.com/paper/2609.21322/) (2026)

## Abstract (arXiv)

How to efficiently utilize the temporal features is crucial, yet challenging, for video restoration. The temporal features usually contain various noisy and uncorrelated information, and they may interfere with the restoration of the current frame. This paper proposes learning noise-robust feature representations to help video restoration. We are inspired by that the neural codec is a natural denoiser. In neural codec, the noisy and uncorrelated contents which are hard to predict but cost lots of bits are more inclined to be discarded for bitrate saving. Therefore, we design a neural compression module to filter the noise and keep the most useful information in features for video restoration. To achieve robustness to noise, our compression module adopts a spatial channel-wise quantization mechanism to adaptively determine the quantization step size for each position in the latent. Experiments show that our method can significantly boost the performance on video denoising, where we obtain 0.13 dB improvement over BasicVSR++ with only 0.23x FLOPs. Meanwhile, our method also obtains SOTA results on video deraining and dehazing.