# Enhancing Video Super-Resolution via Implicit Resampling-based Alignment

arXiv: https://arxiv.org/abs/2305.00163 · HTML: https://mlsanity.com/paper/2305.00163/

## Key facts

- Cited by 35 papers in the map (data through 14 Oct 2026).
- Builds on 22 of its 34 references that are in the map.
- First posted on arXiv 29 Apr 2023.

## Builds on (most cited first, 22 of 22)

- [RAFT: Recurrent All-Pairs Field Transforms for Optical Flow](https://mlsanity.com/paper/2003.12039/) (2020)
- [On the Spectral Bias of Neural Networks](https://mlsanity.com/paper/1806.08734/) (2018)
- [Video Enhancement with Task-Oriented Flow](https://mlsanity.com/paper/1711.09078/) (2017)
- [Learning Continuous Image Representation with Local Implicit Image Function](https://mlsanity.com/paper/2012.09161/) (2020)
- [EDVR: Video Restoration with Enhanced Deformable Convolutional Networks](https://mlsanity.com/paper/1905.02716/) (2019)
- [Optical Flow Estimation using a Spatial Pyramid Network](https://mlsanity.com/paper/1611.00850/) (2016)
- [BasicVSR++: Improving Video Super-Resolution with Enhanced Propagation and Alignment](https://mlsanity.com/paper/2104.13371/) (2021)
- [BasicVSR: The Search for Essential Components in Video Super-Resolution and Beyond](https://mlsanity.com/paper/2012.02181/) (2020)
- [Softmax Splatting for Video Frame Interpolation](https://mlsanity.com/paper/2003.05534/) (2020)
- [VRT: A Video Restoration Transformer](https://mlsanity.com/paper/2201.12288/) (2022)
- [Recurrent Video Restoration Transformer with Guided Deformable Attention](https://mlsanity.com/paper/2206.02146/) (2022)
- [Recurrent Back-Projection Network for Video Super-Resolution](https://mlsanity.com/paper/1903.10128/) (2019)
- [Investigating Tradeoffs in Real-World Video Super-Resolution](https://mlsanity.com/paper/2111.12704/) (2021)
- [Video Super-Resolution Transformer](https://mlsanity.com/paper/2106.06847/) (2021)
- [Video Super-Resolution with Recurrent Structure-Detail Network](https://mlsanity.com/paper/2008.00455/) (2020)
- [Neural Scene Flow Prior](https://mlsanity.com/paper/2111.01253/) (2021)
- [MuCAN: Multi-Correspondence Aggregation Network for Video Super-Resolution](https://mlsanity.com/paper/2007.11803/) (2020)
- [Rethinking Alignment in Video Super-Resolution Transformers](https://mlsanity.com/paper/2207.08494/) (2022)
- [Efficient Video Super-Resolution through Recurrent Latent Space Propagation](https://mlsanity.com/paper/1909.08080/) (2019)
- [UltraSR: Spatial Encoding is a Missing Key for Implicit Image Function-based Arbitrary-Scale Super-Resolution](https://mlsanity.com/paper/2103.12716/) (2021)
- [SSD-GAN: Measuring the Realness in the Spatial and Spectral Domains](https://mlsanity.com/paper/2012.05535/) (2020)
- [Unsupervised Flow-Aligned Sequence-to-Sequence Learning for Video Restoration](https://mlsanity.com/paper/2205.10195/) (2022)

## Built on by (most cited first, 30 of 35)

- [VideoGigaGAN: Towards Detail-rich Video Super-Resolution](https://mlsanity.com/paper/2404.12388/) (2024)
- [Video Super-Resolution Transformer with Masked Inter&Intra-Frame Attention](https://mlsanity.com/paper/2401.06312/) (2024)
- [PMQ-VE: Progressive Multi-Frame Quantization for Video Enhancement](https://mlsanity.com/paper/2505.12266/) (2025)
- [Stream-DiffVSR: Low-Latency Streamable Video Super-Resolution via Auto-Regressive Diffusion](https://mlsanity.com/paper/2512.23709/) (2025)
- [Event-Enhanced Blurry Video Super-Resolution](https://mlsanity.com/paper/2504.13042/) (2025)
- [VSRM: A Robust Mamba-Based Framework for Video Super-Resolution](https://mlsanity.com/paper/2506.22762/) (2025)
- [Enhancing Unregistered Hyperspectral Image Super-Resolution via Unmixing-based Abundance Fusion Learning](https://mlsanity.com/paper/2603.07918/) (2026)
- [Blind Video Super-Resolution based on Implicit Kernels](https://mlsanity.com/paper/2503.07856/) (2025)
- [Space-Time Video Super-resolution with Neural Operator](https://mlsanity.com/paper/2404.06036/) (2024)
- [Rethinking Video Super-Resolution: Towards Diffusion-Based Methods without Motion Alignment](https://mlsanity.com/paper/2503.03355/) (2025)
- [MedVSR: Medical Video Super-Resolution with Cross State-Space Propagation](https://mlsanity.com/paper/2509.21265/) (2025)
- [SR+Codec: a Benchmark of Super-Resolution for Video Compression Bitrate Reduction](https://mlsanity.com/paper/2305.04844/) (2023)
- [Collaborative Feedback Discriminative Propagation for Video Super-Resolution](https://mlsanity.com/paper/2404.04745/) (2024)
- [Spatio-Temporal Distortion Aware Omnidirectional Video Super-Resolution](https://mlsanity.com/paper/2410.11506/) (2024)
- [FCVSR: A Frequency-aware Method for Compressed Video Super-Resolution](https://mlsanity.com/paper/2502.06431/) (2025)
- [Joint Flow And Feature Refinement Using Attention For Video Restoration](https://mlsanity.com/paper/2505.16434/) (2025)
- [MF-LPR2: Multi-Frame License Plate Image Restoration and Recognition using Optical Flow](https://mlsanity.com/paper/2508.14797/) (2025)
- [Rethinking Diffusion Model-Based Video Super-Resolution: Leveraging Dense Guidance from Aligned Features](https://mlsanity.com/paper/2511.16928/) (2025)
- [Seeing the Unseen: Zooming in the Dark with Event Cameras](https://mlsanity.com/paper/2601.02206/) (2026)
- [AIM 2024 Challenge on Video Super-Resolution Quality Assessment: Methods and Results](https://mlsanity.com/paper/2410.04225/) (2024)
- [FIPER: Factorized Features for Robust Image Super-Resolution and Compression](https://mlsanity.com/paper/2410.18083/) (2024)
- [Implicit Neural Representation for Video Restoration](https://mlsanity.com/paper/2506.05488/) (2025)
- [A Systematic Investigation on Deep Learning-Based Omnidirectional Image and Video Super-Resolution](https://mlsanity.com/paper/2506.06710/) (2025)
- [Trajectory-aware Shifted State Space Models for Online Video Super-Resolution](https://mlsanity.com/paper/2508.10453/) (2025)
- [HATIR: Heat-Aware Diffusion for Turbulent Infrared Video Super-Resolution](https://mlsanity.com/paper/2601.04682/) (2026)
- [FedVSR: Towards Model-Agnostic Federated Learning in Video Super-Resolution](https://mlsanity.com/paper/2503.13745/) (2025)
- [DualX-VSR: Dual Axial Spatial×Temporal Transformer for Real-World Video Super-Resolution without Motion Compensation](https://mlsanity.com/paper/2506.04830/) (2025)
- [MambaVSR: Content-Aware Scanning State Space Model for Video Super-Resolution](https://mlsanity.com/paper/2506.11768/) (2025)
- [Gather-Scatter Mamba: Accelerating Propagation with Efficient State Space Model](https://mlsanity.com/paper/2510.00862/) (2025)
- [FMA-Net++: Motion- and Exposure-Aware Joint Video Super-Resolution and Deblurring](https://mlsanity.com/paper/2512.04390/) (2025)

## Abstract (arXiv)

In video super-resolution, it is common to use a frame-wise alignment to support the propagation of information over time. The role of alignment is well-studied for low-level enhancement in video, but existing works overlook a critical step -- resampling. We show through extensive experiments that for alignment to be effective, the resampling should preserve the reference frequency spectrum while minimizing spatial distortions. However, most existing works simply use a default choice of bilinear interpolation for resampling even though bilinear interpolation has a smoothing effect and hinders super-resolution. From these observations, we propose an implicit resampling-based alignment. The sampling positions are encoded by a sinusoidal positional encoding, while the value is estimated with a coordinate network and a window-based cross-attention. We show that bilinear interpolation inherently attenuates high-frequency information while an MLP-based coordinate network can approximate more frequencies. Experiments on synthetic and real-world datasets show that alignment with our proposed implicit resampling enhances the performance of state-of-the-art frameworks with minimal impact on both compute and parameters.