# Frame-Recurrent Video Super-Resolution

arXiv: https://arxiv.org/abs/1801.04590 · HTML: https://mlsanity.com/paper/1801.04590/

## Key facts

- Cited by 160 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 14 Jan 2018.

## Built on by (most cited first, 30 of 160)

- [BasicVSR++: Improving Video Super-Resolution with Enhanced Propagation and Alignment](https://mlsanity.com/paper/2104.13371/) (2021)
- [FlowFormer: A Transformer Architecture for Optical Flow](https://mlsanity.com/paper/2203.16194/) (2022)
- [VRT: A Video Restoration Transformer](https://mlsanity.com/paper/2201.12288/) (2022)
- [Recurrent Video Restoration Transformer with Guided Deformable Attention](https://mlsanity.com/paper/2206.02146/) (2022)
- [Efficient Long-Range Attention Network for Image Super-resolution](https://mlsanity.com/paper/2203.06697/) (2022)
- [Video Super-Resolution Transformer](https://mlsanity.com/paper/2106.06847/) (2021)
- [FlowFormer++: Masked Cost Volume Autoencoding for Pretraining Optical Flow Estimation](https://mlsanity.com/paper/2303.01237/) (2023)
- [Generating Long Videos of Dynamic Scenes](https://mlsanity.com/paper/2206.03429/) (2022)
- [VideoFlow: Exploiting Temporal Cues for Multi-frame Optical Flow Estimation](https://mlsanity.com/paper/2303.08340/) (2023)
- [Rethinking Alignment in Video Super-Resolution Transformers](https://mlsanity.com/paper/2207.08494/) (2022)
- [SeedVR: Seeding Infinity in Diffusion Transformer Towards Generic Video Restoration](https://mlsanity.com/paper/2501.01320/) (2025)
- [Learning Trajectory-Aware Transformer for Video Super-Resolution](https://mlsanity.com/paper/2204.04216/) (2022)
- [STAR: Spatial-Temporal Augmentation with Text-to-Video Models for Real-World Video Super-Resolution](https://mlsanity.com/paper/2501.02976/) (2025)
- [RSTT: Real-time Spatial Temporal Transformer for Space-Time Video Super-Resolution](https://mlsanity.com/paper/2203.14186/) (2022)
- [Frame-Event Alignment and Fusion Network for High Frame Rate Tracking](https://mlsanity.com/paper/2305.15688/) (2023)
- [DL4SciVis: A State-of-the-Art Survey on Deep Learning for Scientific Visualization](https://mlsanity.com/paper/2204.06504/) (2022)
- [Omniscient Video Super-Resolution](https://mlsanity.com/paper/2103.15683/) (2021)
- [VideoGigaGAN: Towards Detail-rich Video Super-Resolution](https://mlsanity.com/paper/2404.12388/) (2024)
- [Mitigating Artifacts in Real-World Video Super-Resolution Models](https://mlsanity.com/paper/2212.07339/) (2022)
- [Video Super-Resolution Transformer with Masked Inter&Intra-Frame Attention](https://mlsanity.com/paper/2401.06312/) (2024)
- [FMA-Net: Flow-Guided Dynamic Filtering and Iterative Feature Refinement with Multi-Attention for Joint Video Super-Resolution and Deblurring](https://mlsanity.com/paper/2401.03707/) (2024)
- [No More Strided Convolutions or Pooling: A New CNN Building Block for Low-Resolution Images and Small Objects](https://mlsanity.com/paper/2208.03641/) (2022)
- [COMISR: Compression-Informed Video Super-Resolution](https://mlsanity.com/paper/2105.01237/) (2021)
- [Learning Spatiotemporal Frequency-Transformer for Compressed Video Super-Resolution](https://mlsanity.com/paper/2208.03012/) (2022)
- [Video Demoireing with Relation-Based Temporal Consistency](https://mlsanity.com/paper/2204.02957/) (2022)
- [Unidirectional Video Denoising by Mimicking Backward Recurrent Modules with Look-ahead Forward Ones](https://mlsanity.com/paper/2204.05532/) (2022)
- [Variational Deep Image Restoration](https://mlsanity.com/paper/2207.01074/) (2022)
- [AnimeSR: Learning Real-World Super-Resolution Models for Animation Videos](https://mlsanity.com/paper/2206.07038/) (2022)
- [Learning Spatial Adaptation and Temporal Coherence in Diffusion Models for Video Super-Resolution](https://mlsanity.com/paper/2403.17000/) (2024)
- [Can SAM Boost Video Super-Resolution?](https://mlsanity.com/paper/2305.06524/) (2023)

## Abstract (arXiv)

Recent advances in video super-resolution have shown that convolutional neural networks combined with motion compensation are able to merge information from multiple low-resolution (LR) frames to generate high-quality images. Current state-of-the-art methods process a batch of LR frames to generate a single high-resolution (HR) frame and run this scheme in a sliding window fashion over the entire video, effectively treating the problem as a large number of separate multi-frame super-resolution tasks. This approach has two main weaknesses: 1) Each input frame is processed and warped multiple times, increasing the computational cost, and 2) each output frame is estimated independently conditioned on the input frames, limiting the system's ability to produce temporally consistent results. In this work, we propose an end-to-end trainable frame-recurrent video super-resolution framework that uses the previously inferred HR estimate to super-resolve the subsequent frame. This naturally encourages temporally consistent results and reduces the computational cost by warping only one image in each step. Furthermore, due to its recurrent nature, the proposed method has the ability to assimilate a large number of previous frames without increased computational demands. Extensive evaluations and comparisons with previous methods validate the strengths of our approach and demonstrate that the proposed framework is able to significantly outperform the current state of the art.