# Lightweight Hybrid Video Compression Framework Using Reference-Guided Restoration Network

arXiv: https://arxiv.org/abs/2303.11592 · HTML: https://mlsanity.com/paper/2303.11592/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 26 of its 53 references that are in the map.
- First posted on arXiv 21 Mar 2023.

## Builds on (most cited first, 26 of 26)

- [Deformable Convolutional Networks](https://mlsanity.com/paper/1703.06211/) (2017)
- [Beyond a Gaussian Denoiser: Residual Learning of Deep CNN for Image Denoising](https://mlsanity.com/paper/1608.03981/) (2016)
- [PWC-Net: CNNs for Optical Flow Using Pyramid, Warping, and Cost Volume](https://mlsanity.com/paper/1709.02371/) (2017)
- [DVC: An End-to-end Deep Video Compression Framework](https://mlsanity.com/paper/1812.00101/) (2018)
- [Deep Contextual Video Compression](https://mlsanity.com/paper/2109.15047/) (2021)
- [FVC: A New Framework towards Deep Video Compression in Feature Space](https://mlsanity.com/paper/2105.09600/) (2021)
- [Video Compression through Image Interpolation](https://mlsanity.com/paper/1804.06919/) (2018)
- [Compression Artifacts Reduction by a Deep Convolutional Network](https://mlsanity.com/paper/1504.06993/) (2015)
- [M-LVC: Multiple Frames Prediction for Learned Video Compression](https://mlsanity.com/paper/2004.10290/) (2020)
- [Learning for Video Compression with Hierarchical Quality and Recurrent Enhancement](https://mlsanity.com/paper/2003.01966/) (2020)
- [Content Adaptive and Error Propagation Aware Deep Video Compression](https://mlsanity.com/paper/2003.11282/) (2020)
- [ELF-VC: Efficient Learned Flexible-Rate Video Coding](https://mlsanity.com/paper/2104.14335/) (2021)
- [Robust Reference-based Super-Resolution via C2-Matching](https://mlsanity.com/paper/2106.01863/) (2021)
- [Improving Deep Video Compression by Resolution-adaptive Flow Coding](https://mlsanity.com/paper/2009.05982/) (2020)
- [Coarse-to-fine Deep Video Coding with Hyperprior-guided Mode Prediction](https://mlsanity.com/paper/2206.07460/) (2022)
- [MFQE 2.0: A New Approach for Multi-frame Quality Enhancement on Compressed Video](https://mlsanity.com/paper/1902.09707/) (2019)
- [Multi-Frame Quality Enhancement for Compressed Video](https://mlsanity.com/paper/1803.04680/) (2018)
- [Enhancing Quality for HEVC Compressed Videos](https://mlsanity.com/paper/1709.06734/) (2017)
- [A Convolutional Neural Network Approach for Post-Processing in HEVC Intra Coding](https://mlsanity.com/paper/1608.06690/) (2016)
- [Deep Generative Video Compression](https://mlsanity.com/paper/1810.02845/) (2018)
- [Non-Local ConvLSTM for Video Compression Artifact Reduction](https://mlsanity.com/paper/1910.12286/) (2019)
- [Learned Video Compression with Feature-level Residuals](https://mlsanity.com/paper/2004.08283/) (2020)
- [DPW-SDNet: Dual Pixel-Wavelet Domain Deep CNNs for Soft Decoding of JPEG-Compressed Images](https://mlsanity.com/paper/1805.10558/) (2018)
- [Partition-Aware Adaptive Switching Neural Networks for Post-Processing in HEVC](https://mlsanity.com/paper/1912.11604/) (2019)
- [Neural network-based arithmetic coding of intra prediction modes in HEVC](https://mlsanity.com/paper/1709.05737/) (2017)
- [Boosting the Performance of Video Compression Artifact Reduction with Reference Frame Proposals and Frequency Domain Information](https://mlsanity.com/paper/2105.14962/) (2021)

## Abstract (arXiv)

Recent deep-learning-based video compression methods brought coding gains over conventional codecs such as AVC and HEVC. However, learning-based codecs generally require considerable computation time and model complexity. In this paper, we propose a new lightweight hybrid video codec consisting of a conventional video codec(HEVC / VVC), a lossless image codec, and our new restoration network. Precisely, our encoder consists of the conventional video encoder and a lossless image encoder, transmitting a lossy-compressed video bitstream along with a losslessly-compressed reference frame. The decoder is constructed with corresponding video/image decoders and a new restoration network, which enhances the compressed video in two-step processes. In the first step, a network trained with a large video dataset restores the details lost by the conventional encoder. Then, we further boost the video quality with the guidance of a reference image, which is a losslessly compressed video frame. The reference image provides video-specific information, which can be utilized to better restore the details of a compressed video. Experimental results show that the proposed method achieves comparable performance to top-tier methods, even when applied to HEVC. Nevertheless, our method has lower complexity, a faster run time, and can be easily integrated into existing conventional codecs.