# TDAN: Temporally Deformable Alignment Network for Video Super-Resolution

arXiv: https://arxiv.org/abs/1812.02898 · HTML: https://mlsanity.com/paper/1812.02898/

## Key facts

- Cited by 226 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 7 Dec 2018.

## Built on by (most cited first, 30 of 226)

- [BasicVSR++: Improving Video Super-Resolution with Enhanced Propagation and Alignment](https://mlsanity.com/paper/2104.13371/) (2021)
- [VRT: A Video Restoration Transformer](https://mlsanity.com/paper/2201.12288/) (2022)
- [Recurrent Video Restoration Transformer with Guided Deformable Attention](https://mlsanity.com/paper/2206.02146/) (2022)
- [Neural Video Compression with Diverse Contexts](https://mlsanity.com/paper/2302.14402/) (2023)
- [FVC: A New Framework towards Deep Video Compression in Feature Space](https://mlsanity.com/paper/2105.09600/) (2021)
- [XVFI: eXtreme Video Frame Interpolation](https://mlsanity.com/paper/2103.16206/) (2021)
- [Video Super-Resolution Transformer](https://mlsanity.com/paper/2106.06847/) (2021)
- [ALIKED: A Lighter Keypoint and Descriptor Extraction Network via Deformable Transformation](https://mlsanity.com/paper/2304.03608/) (2023)
- [Towards An End-to-End Framework for Flow-Guided Video Inpainting](https://mlsanity.com/paper/2204.02663/) (2022)
- [VideoINR: Learning Video Implicit Neural Representation for Continuous Space-Time Super-Resolution](https://mlsanity.com/paper/2206.04647/) (2022)
- [FaPN: Feature-aligned Pyramid Network for Dense Image Prediction](https://mlsanity.com/paper/2108.07058/) (2021)
- [Changer: Feature Interaction is What You Need for Change Detection](https://mlsanity.com/paper/2209.08290/) (2022)
- [Rethinking Alignment in Video Super-Resolution Transformers](https://mlsanity.com/paper/2207.08494/) (2022)
- [DG-Font: Deformable Generative Networks for Unsupervised Font Generation](https://mlsanity.com/paper/2104.03064/) (2021)
- [A Simple Baseline for Video Restoration with Grouped Spatial-temporal Shift](https://mlsanity.com/paper/2206.10810/) (2022)
- [Real-World Single Image Super-Resolution: A Brief Review](https://mlsanity.com/paper/2103.02368/) (2021)
- [Burst Image Restoration and Enhancement](https://mlsanity.com/paper/2110.03680/) (2021)
- [SeedVR: Seeding Infinity in Diffusion Transformer Towards Generic Video Restoration](https://mlsanity.com/paper/2501.01320/) (2025)
- [Learning Trajectory-Aware Transformer for Video Super-Resolution](https://mlsanity.com/paper/2204.04216/) (2022)
- [Temporal Modulation Network for Controllable Space-Time Video Super-Resolution](https://mlsanity.com/paper/2104.10642/) (2021)
- [Implicit Temporal Modeling with Learnable Alignment for Video Recognition](https://mlsanity.com/paper/2304.10465/) (2023)
- [HDR Video Reconstruction: A Coarse-to-fine Network and A Real-world Benchmark Dataset](https://mlsanity.com/paper/2103.14943/) (2021)
- [RSTT: Real-time Spatial Temporal Transformer for Space-Time Video Super-Resolution](https://mlsanity.com/paper/2203.14186/) (2022)
- [Frame-Event Alignment and Fusion Network for High Frame Rate Tracking](https://mlsanity.com/paper/2305.15688/) (2023)
- [DiffIR2VR-Zero: Zero-Shot Video Restoration with Diffusion-based Image Restoration Models](https://mlsanity.com/paper/2407.01519/) (2024)
- [Learning Spatial-Temporal Implicit Neural Representations for Event-Guided Video Super-Resolution](https://mlsanity.com/paper/2303.13767/) (2023)
- [Burstormer: Burst Image Restoration and Enhancement Transformer](https://mlsanity.com/paper/2304.01194/) (2023)
- [Omniscient Video Super-Resolution](https://mlsanity.com/paper/2103.15683/) (2021)
- [VideoGigaGAN: Towards Detail-rich Video Super-Resolution](https://mlsanity.com/paper/2404.12388/) (2024)
- [Mitigating Artifacts in Real-World Video Super-Resolution Models](https://mlsanity.com/paper/2212.07339/) (2022)

## Abstract (arXiv)

Video super-resolution (VSR) aims to restore a photo-realistic high-resolution (HR) video frame from both its corresponding low-resolution (LR) frame (reference frame) and multiple neighboring frames (supporting frames). Due to varying motion of cameras or objects, the reference frame and each support frame are not aligned. Therefore, temporal alignment is a challenging yet important problem for VSR. Previous VSR methods usually utilize optical flow between the reference frame and each supporting frame to wrap the supporting frame for temporal alignment. Therefore, the performance of these image-level wrapping-based models will highly depend on the prediction accuracy of optical flow, and inaccurate optical flow will lead to artifacts in the wrapped supporting frames, which also will be propagated into the reconstructed HR video frame. To overcome the limitation, in this paper, we propose a temporal deformable alignment network (TDAN) to adaptively align the reference frame and each supporting frame at the feature level without computing optical flow. The TDAN uses features from both the reference frame and each supporting frame to dynamically predict offsets of sampling convolution kernels. By using the corresponding kernels, TDAN transforms supporting frames to align with the reference frame. To predict the HR video frame, a reconstruction network taking aligned frames and the reference frame is utilized. Experimental results demonstrate the effectiveness of the proposed TDAN-based VSR model.