# Revisiting Temporal Alignment for Video Restoration

arXiv: https://arxiv.org/abs/2111.15288 · HTML: https://mlsanity.com/paper/2111.15288/

## Key facts

- Cited by 24 papers in the map (data through 14 Oct 2026).
- Builds on 26 of its 42 references that are in the map.
- First posted on arXiv 30 Nov 2021.

## Builds on (most cited first, 26 of 26)

- [RAFT: Recurrent All-Pairs Field Transforms for Optical Flow](https://mlsanity.com/paper/2003.12039/) (2020)
- [Deformable Convolutional Networks](https://mlsanity.com/paper/1703.06211/) (2017)
- [Image Super-Resolution Using Very Deep Residual Channel Attention Networks](https://mlsanity.com/paper/1807.02758/) (2018)
- [Video Enhancement with Task-Oriented Flow](https://mlsanity.com/paper/1711.09078/) (2017)
- [EDVR: Video Restoration with Enhanced Deformable Convolutional Networks](https://mlsanity.com/paper/1905.02716/) (2019)
- [Optical Flow Estimation using a Spatial Pyramid Network](https://mlsanity.com/paper/1611.00850/) (2016)
- [BasicVSR++: Improving Video Super-Resolution with Enhanced Propagation and Alignment](https://mlsanity.com/paper/2104.13371/) (2021)
- [BasicVSR: The Search for Essential Components in Video Super-Resolution and Beyond](https://mlsanity.com/paper/2012.02181/) (2020)
- [TDAN: Temporally Deformable Alignment Network for Video Super-Resolution](https://mlsanity.com/paper/1812.02898/) (2018)
- [Video Object Segmentation with Language Referring Expressions](https://mlsanity.com/paper/1803.08006/) (2018)
- [Recurrent Back-Projection Network for Video Super-Resolution](https://mlsanity.com/paper/1903.10128/) (2019)
- [FastDVDnet: Towards Real-Time Deep Video Denoising Without Flow Estimation](https://mlsanity.com/paper/1907.01361/) (2019)
- [Video Super-Resolution Transformer](https://mlsanity.com/paper/2106.06847/) (2021)
- [Video Super-Resolution with Recurrent Structure-Detail Network](https://mlsanity.com/paper/2008.00455/) (2020)
- [Spatio-Temporal Filter Adaptive Network for Video Deblurring](https://mlsanity.com/paper/1904.12257/) (2019)
- [MuCAN: Multi-Correspondence Aggregation Network for Video Super-Resolution](https://mlsanity.com/paper/2007.11803/) (2020)
- [LAPAR: Linearly-Assembled Pixel-Adaptive Regression Network for Single Image Super-Resolution and Beyond](https://mlsanity.com/paper/2105.10422/) (2021)
- [Handheld Multi-Frame Super-Resolution](https://mlsanity.com/paper/1905.03277/) (2019)
- [Cascaded Deep Video Deblurring Using Temporal Sharpness Prior](https://mlsanity.com/paper/2004.02501/) (2020)
- [ARVo: Learning All-Range Volumetric Correspondence for Video Deblurring](https://mlsanity.com/paper/2103.04260/) (2021)
- [Online Video Deblurring via Dynamic Temporal Blending Network](https://mlsanity.com/paper/1704.03285/) (2017)
- [Best-Buddy GANs for Highly Detailed Image Super-Resolution](https://mlsanity.com/paper/2103.15295/) (2021)
- [RAISR: Rapid and Accurate Image Super Resolution](https://mlsanity.com/paper/1606.01299/) (2016)
- [Learning Deformable Kernels for Image and Video Denoising](https://mlsanity.com/paper/1904.06903/) (2019)
- [Image Super-Resolution via Attention based Back Projection Networks](https://mlsanity.com/paper/1910.04476/) (2019)
- [DAVID: Dual-Attentional Video Deblurring](https://mlsanity.com/paper/1912.03445/) (2019)

## Built on by (most cited first, 24 of 24)

- [NeRFLiX: High-Quality Neural View Synthesis by Learning a Degradation-Driven Inter-viewpoint MiXer](https://mlsanity.com/paper/2303.06919/) (2023)
- [Video Adverse-Weather-Component Suppression Network via Weather Messenger and Adversarial Backpropagation](https://mlsanity.com/paper/2309.13700/) (2023)
- [Real-RawVSR: Real-World Raw Video Super-Resolution with a Benchmark Dataset](https://mlsanity.com/paper/2209.12475/) (2022)
- [Exploring Motion Ambiguity and Alignment for High-Quality Video Frame Interpolation](https://mlsanity.com/paper/2203.10291/) (2022)
- [Genuine Knowledge from Practice: Diffusion Test-Time Adaptation for Video Adverse Weather Removal](https://mlsanity.com/paper/2403.07684/) (2024)
- [Recurrent Self-Supervised Video Denoising with Denser Receptive Field](https://mlsanity.com/paper/2308.03608/) (2023)
- [TMP: Temporal Motion Propagation for Online Video Super-Resolution](https://mlsanity.com/paper/2312.09909/) (2023)
- [A Comprehensive Study of Object Tracking in Low-Light Environments](https://mlsanity.com/paper/2312.16250/) (2023)
- [EvTexture: Event-driven Texture Enhancement for Video Super-Resolution](https://mlsanity.com/paper/2406.13457/) (2024)
- [BVI-RLV: A Fully Registered Dataset for Low-Light Video Enhancement](https://mlsanity.com/paper/2407.03535/) (2024)
- [Domain-adaptive Video Deblurring via Test-time Blurring](https://mlsanity.com/paper/2407.09059/) (2024)
- [JNMR: Joint Non-linear Motion Regression for Video Frame Interpolation](https://mlsanity.com/paper/2206.04231/) (2022)
- [From NeRFLiX to NeRFLiX++: A General NeRF-Agnostic Restorer Paradigm](https://mlsanity.com/paper/2306.06388/) (2023)
- [Zero-TIG: Temporal Consistency-Aware Zero-Shot Illumination-Guided Low-light Video Enhancement](https://mlsanity.com/paper/2503.11175/) (2025)
- [BVI-Lowlight: Fully Registered Benchmark Dataset for Low-Light Video Enhancement](https://mlsanity.com/paper/2402.01970/) (2024)
- [DeLiVR: Differential Spatiotemporal Lie Bias for Efficient Video Deraining](https://mlsanity.com/paper/2509.21719/) (2025)
- [Reference-based Restoration of Digitized Analog Videotapes](https://mlsanity.com/paper/2310.14926/) (2023)
- [TurboVSR: Fantastic Video Upscalers and Where to Find Them](https://mlsanity.com/paper/2506.23618/) (2025)
- [BVI-Mamba: Video Enhancement Using a Visual State-Space Model for Low-Light and Underwater Environments](https://mlsanity.com/paper/2604.23655/) (2026)
- [Towards Robust and Generalizable Continuous Space-Time Video Super-Resolution with Events](https://mlsanity.com/paper/2510.03833/) (2025)
- [AbsoluteDegradation: A Physics-Inspired Synthetic Film-Degradation Pipeline and Archival Film Restoration Benchmark](https://mlsanity.com/paper/2607.02131/) (2026)
- [ViStripformer: A Token-Efficient Transformer for Versatile Video Restoration](https://mlsanity.com/paper/2312.14502/) (2023)
- [Joint Depth and Reflectivity Estimation using Single-Photon LiDAR](https://mlsanity.com/paper/2505.13250/) (2025)
- [EvTexture++: Event-Driven Texture Enhancement for Video Super-Resolution](https://mlsanity.com/paper/2606.13580/) (2026)

## Abstract (arXiv)

Long-range temporal alignment is critical yet challenging for video restoration tasks. Recently, some works attempt to divide the long-range alignment into several sub-alignments and handle them progressively. Although this operation is helpful in modeling distant correspondences, error accumulation is inevitable due to the propagation mechanism. In this work, we present a novel, generic iterative alignment module which employs a gradual refinement scheme for sub-alignments, yielding more accurate motion compensation. To further enhance the alignment accuracy and temporal consistency, we develop a non-parametric re-weighting method, where the importance of each neighboring frame is adaptively evaluated in a spatial-wise way for aggregation. By virtue of the proposed strategies, our model achieves state-of-the-art performance on multiple benchmarks across a range of video restoration tasks including video super-resolution, denoising and deblurring. Our project is available in urlhttps://github.com/redrock303/Revisiting-Temporal-Alignment-for-Video-Restoration.git.