# Deep Video Super-Resolution using HR Optical Flow Estimation

arXiv: https://arxiv.org/abs/2001.02129 · HTML: https://mlsanity.com/paper/2001.02129/

## Key facts

- Cited by 40 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 6 Jan 2020.

## Built on by (most cited first, 30 of 40)

- [VRT: A Video Restoration Transformer](https://mlsanity.com/paper/2201.12288/) (2022)
- [Recurrent Video Restoration Transformer with Guided Deformable Attention](https://mlsanity.com/paper/2206.02146/) (2022)
- [MonoNeRF: Learning a Generalizable Dynamic Radiance Field from Monocular Videos](https://mlsanity.com/paper/2212.13056/) (2022)
- [Learning Spatial-Temporal Implicit Neural Representations for Event-Guided Video Super-Resolution](https://mlsanity.com/paper/2303.13767/) (2023)
- [Lightweight Event-based Optical Flow Estimation via Iterative Deblurring](https://mlsanity.com/paper/2211.13726/) (2022)
- [Optical Flow Estimation for Spiking Camera](https://mlsanity.com/paper/2110.03916/) (2021)
- [FMA-Net: Flow-Guided Dynamic Filtering and Iterative Feature Refinement with Multi-Attention for Joint Video Super-Resolution and Deblurring](https://mlsanity.com/paper/2401.03707/) (2024)
- [Learning Spatial Adaptation and Temporal Coherence in Diffusion Models for Video Super-Resolution](https://mlsanity.com/paper/2403.17000/) (2024)
- [NTIRE 2022 Challenge on Super-Resolution and Quality Enhancement of Compressed Video: Dataset, Methods and Results](https://mlsanity.com/paper/2204.09314/) (2022)
- [Benchmark Dataset and Effective Inter-Frame Alignment for Real-World Video Super-Resolution](https://mlsanity.com/paper/2212.05342/) (2022)
- [Local-Global Temporal Difference Learning for Satellite Video Super-Resolution](https://mlsanity.com/paper/2304.04421/) (2023)
- [Detecting and Tracking Small and Dense Moving Objects in Satellite Videos: A Benchmark](https://mlsanity.com/paper/2111.12960/) (2021)
- [AnyFlow: Arbitrary Scale Optical Flow with Implicit Neural Representation](https://mlsanity.com/paper/2303.16493/) (2023)
- [A Perturbation-Constrained Adversarial Attack for Evaluating the Robustness of Optical Flow](https://mlsanity.com/paper/2203.13214/) (2022)
- [LAVIB: A Large-scale Video Interpolation Benchmark](https://mlsanity.com/paper/2406.09754/) (2024)
- [GyroFlow+: Gyroscope-Guided Unsupervised Deep Homography and Optical Flow Learning](https://mlsanity.com/paper/2301.10018/) (2023)
- [Infrared Small Target Detection in Satellite Videos: A New Dataset and A Novel Recurrent Feature Refinement Framework](https://mlsanity.com/paper/2409.12448/) (2024)
- [Local Motion and Contrast Priors Driven Deep Network for Infrared Small Target Super-Resolution](https://mlsanity.com/paper/2201.01014/) (2022)
- [TMP: Temporal Motion Propagation for Online Video Super-Resolution](https://mlsanity.com/paper/2312.09909/) (2023)
- [EvTexture: Event-driven Texture Enhancement for Video Super-Resolution](https://mlsanity.com/paper/2406.13457/) (2024)
- [Dense Dual-Attention Network for Light Field Image Super-Resolution](https://mlsanity.com/paper/2110.12114/) (2021)
- [NTIRE 2023 Challenge on Light Field Image Super-Resolution: Dataset, Methods and Results](https://mlsanity.com/paper/2304.10415/) (2023)
- [HR-INR: Continuous Space-Time Video Super-Resolution via Event Camera](https://mlsanity.com/paper/2405.13389/) (2024)
- [Towards True Detail Restoration for Super-Resolution: A Benchmark and a Quality Metric](https://mlsanity.com/paper/2203.08923/) (2022)
- [An Efficient Network Design for Face Video Super-resolution](https://mlsanity.com/paper/2109.13626/) (2021)
- [Optical Flow for Video Super-Resolution: A Survey](https://mlsanity.com/paper/2203.10462/) (2022)
- [UIERL: Internal-External Representation Learning Network for Underwater Image Enhancement](https://mlsanity.com/paper/2306.08344/) (2023)
- [An Operational Deep Learning System for Satellite-Based High-Resolution Global Nowcasting](https://mlsanity.com/paper/2510.13050/) (2025)
- [Weakly-supervised Representation Learning for Video Alignment and Analysis](https://mlsanity.com/paper/2302.04064/) (2023)
- [FlowFormer: A Transformer Architecture and Its Masked Cost Volume Autoencoding for Optical Flow](https://mlsanity.com/paper/2306.05442/) (2023)

## Abstract (arXiv)

Video super-resolution (SR) aims at generating a sequence of high-resolution (HR) frames with plausible and temporally consistent details from their low-resolution (LR) counterparts. The key challenge for video SR lies in the effective exploitation of temporal dependency between consecutive frames. Existing deep learning based methods commonly estimate optical flows between LR frames to provide temporal dependency. However, the resolution conflict between LR optical flows and HR outputs hinders the recovery of fine details. In this paper, we propose an end-to-end video SR network to super-resolve both optical flows and images. Optical flow SR from LR frames provides accurate temporal dependency and ultimately improves video SR performance. Specifically, we first propose an optical flow reconstruction network (OFRnet) to infer HR optical flows in a coarse-to-fine manner. Then, motion compensation is performed using HR optical flows to encode temporal dependency. Finally, compensated LR inputs are fed to a super-resolution network (SRnet) to generate SR results. Extensive experiments have been conducted to demonstrate the effectiveness of HR optical flows for SR performance improvement. Comparative results on the Vid4 and DAVIS-10 datasets show that our network achieves the state-of-the-art performance.