# A Codec Information Assisted Framework for Efficient Compressed Video Super-Resolution

arXiv: https://arxiv.org/abs/2210.08229 · HTML: https://mlsanity.com/paper/2210.08229/

## Key facts

- Cited by 6 papers in the map (data through 14 Oct 2026).
- Builds on 27 of its 37 references that are in the map.
- First posted on arXiv 15 Oct 2022.

## Builds on (most cited first, 27 of 27)

- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [Categorical Reparameterization with Gumbel-Softmax](https://mlsanity.com/paper/1611.01144/) (2016)
- [Deformable Convolutional Networks](https://mlsanity.com/paper/1703.06211/) (2017)
- [PWC-Net: CNNs for Optical Flow Using Pyramid, Warping, and Cost Volume](https://mlsanity.com/paper/1709.02371/) (2017)
- [Deformable ConvNets v2: More Deformable, Better Results](https://mlsanity.com/paper/1811.11168/) (2018)
- [Video Enhancement with Task-Oriented Flow](https://mlsanity.com/paper/1711.09078/) (2017)
- [EDVR: Video Restoration with Enhanced Deformable Convolutional Networks](https://mlsanity.com/paper/1905.02716/) (2019)
- [Deep Laplacian Pyramid Networks for Fast and Accurate Super-Resolution](https://mlsanity.com/paper/1704.03915/) (2017)
- [Optical Flow Estimation using a Spatial Pyramid Network](https://mlsanity.com/paper/1611.00850/) (2016)
- [Recovering Realistic Texture in Image Super-resolution by Deep Spatial Feature Transform](https://mlsanity.com/paper/1804.02815/) (2018)
- [BasicVSR++: Improving Video Super-Resolution with Enhanced Propagation and Alignment](https://mlsanity.com/paper/2104.13371/) (2021)
- [BasicVSR: The Search for Essential Components in Video Super-Resolution and Beyond](https://mlsanity.com/paper/2012.02181/) (2020)
- [TDAN: Temporally Deformable Alignment Network for Video Super-Resolution](https://mlsanity.com/paper/1812.02898/) (2018)
- [BlockDrop: Dynamic Inference Paths in Residual Networks](https://mlsanity.com/paper/1711.08393/) (2017)
- [Frame-Recurrent Video Super-Resolution](https://mlsanity.com/paper/1801.04590/) (2018)
- [Real-Time Video Super-Resolution with Spatio-Temporal Networks and Motion Compensation](https://mlsanity.com/paper/1611.05250/) (2016)
- [Recurrent Back-Projection Network for Video Super-Resolution](https://mlsanity.com/paper/1903.10128/) (2019)
- [Video Super-Resolution with Recurrent Structure-Detail Network](https://mlsanity.com/paper/2008.00455/) (2020)
- [MuCAN: Multi-Correspondence Aggregation Network for Video Super-Resolution](https://mlsanity.com/paper/2007.11803/) (2020)
- [ClassSR: A General Framework to Accelerate Super-Resolution Networks by Data Characteristic](https://mlsanity.com/paper/2103.04039/) (2021)
- [Efficient Video Super-Resolution through Recurrent Latent Space Propagation](https://mlsanity.com/paper/1909.08080/) (2019)
- [Skip-Convolutions for Efficient Video Processing](https://mlsanity.com/paper/2104.11487/) (2021)
- [Omniscient Video Super-Resolution](https://mlsanity.com/paper/2103.15683/) (2021)
- [COMISR: Compression-Informed Video Super-Resolution](https://mlsanity.com/paper/2105.01237/) (2021)
- [Deep Adaptive Inference Networks for Single Image Super-Resolution](https://mlsanity.com/paper/2004.03915/) (2020)
- [FAST: A Framework to Accelerate Super-Resolution Processing on Compressed Videos](https://mlsanity.com/paper/1603.08968/) (2016)
- [Exploring Sparsity in Image Super-Resolution for Efficient Inference](https://mlsanity.com/paper/2006.09603/) (2020)

## Built on by (most cited first, 6 of 6)

- [CPGA: Coding Priors-Guided Aggregation Network for Compressed Video Quality Enhancement](https://mlsanity.com/paper/2403.10362/) (2024)
- [Plug-and-Play Versatile Compressed Video Enhancement](https://mlsanity.com/paper/2504.15380/) (2025)
- [FCVSR: A Frequency-aware Method for Compressed Video Super-Resolution](https://mlsanity.com/paper/2502.06431/) (2025)
- [Compressed-Domain-Aware Online Video Super-Resolution](https://mlsanity.com/paper/2603.07694/) (2026)
- [ANVIL: Accelerator-Native Video Interpolation via Codec Motion Vector Priors](https://mlsanity.com/paper/2603.26835/) (2026)
- [DiffCVE: Diffusion-based Compressed Video Enhancement](https://mlsanity.com/paper/2607.07195/) (2026)

## Abstract (arXiv)

Online processing of compressed videos to increase their resolutions attracts increasing and broad attention. Video Super-Resolution (VSR) using recurrent neural network architecture is a promising solution due to its efficient modeling of long-range temporal dependencies. However, state-of-the-art recurrent VSR models still require significant computation to obtain a good performance, mainly because of the complicated motion estimation for frame/feature alignment and the redundant processing of consecutive video frames. In this paper, considering the characteristics of compressed videos, we propose a Codec Information Assisted Framework (CIAF) to boost and accelerate recurrent VSR models for compressed videos. Firstly, the framework reuses the coded video information of Motion Vectors to model the temporal relationships between adjacent frames. Experiments demonstrate that the models with Motion Vector based alignment can significantly boost the performance with negligible additional computation, even comparable to those using more complex optical flow based alignment. Secondly, by further making use of the coded video information of Residuals, the framework can be informed to skip the computation on redundant pixels. Experiments demonstrate that the proposed framework can save up to 70% of the computation without performance drop on the REDS4 test videos encoded by H.264 when CRF is 23.