mlsanityData to 7 Oct 2026

VRT: A Video Restoration Transformer

arXiv 2201.12288 ↗First posted 28 Jan 2022cs.CV eess.IVcited by 219

Not on the map. Why?

Cited by 44 in Diffusion models, but placed by its own abstract, which didn’t fit.

Abstract (arXiv)

Video restoration (e.g., video super-resolution) aims to restore high-quality frames from low-quality frames. Different from single image restoration, video restoration generally requires to utilize temporal information from multiple adjacent but usually misaligned video frames. Existing deep methods generally tackle with this by exploiting a sliding window strategy or a recurrent architecture, which either is restricted by frame-by-frame restoration or lacks long-range modelling ability. In this paper, we propose a Video Restoration Transformer (VRT) with parallel frame prediction and long-range temporal dependency modelling abilities. More specifically, VRT is composed of multiple scales, each of which consists of two kinds of modules: temporal mutual self attention (TMSA) and parallel warping. TMSA divides the video into small clips, on which mutual attention is applied for joint motion estimation, feature alignment and feature fusion, while self attention is used for feature extraction. To enable cross-clip interactions, the video sequence is shifted for every other layer. Besides, parallel warping is used to further fuse information from neighboring frames by parallel feature warping. Experimental results on five tasks, including video super-resolution, video deblurring, video denoising, video frame interpolation and space-time video super-resolution, demonstrate that VRT outperforms the state-of-the-art methods by large margins (up to 2.16dB) on fourteen benchmark datasets.

Key facts. Cited by 219 papers in the map (data through 7 Oct 2026). Builds on 94 of its 135 references that are in the map. First posted on arXiv 28 Jan 2022.

Builds on

94 of 135 references.

References · 94

  1. 2017Attention Is All You Needcited by 59,932
  2. 2014Adam: A Method for Stochastic Optimizationcited by 36,960
  3. 2020An Image is Worth 16x16 Words: Transformers for Image Recognition at Scalecited by 29,915
  4. 2015U-Net: Convolutional Networks for Biomedical Image Segmentationcited by 19,604
  5. 2021Swin Transformer: Hierarchical Vision Transformer using Shifted Windowscited by 12,082
  6. 2020End-to-End Object Detection with Transformerscited by 7,384
  7. 2016SGDR: Stochastic Gradient Descent with Warm Restartscited by 4,993
  8. 2016Photo-Realistic Single Image Super-Resolution Using a Generative Adversarial Networkcited by 1,898
  9. 2021SwinIR: Image Restoration Using Swin Transformercited by 1,869
  10. 2020GLU Variants Improve Transformercited by 1,724
  11. 2021ViViT: A Video Vision Transformercited by 1,610
  12. 2021Is Space-Time Attention All You Need for Video Understanding?cited by 1,609
38 more
  1. 2017Deformable Convolutional Networkscited by 1,481
  2. 2016Beyond a Gaussian Denoiser: Residual Learning of Deep CNN for Image Denoisingcited by 1,352
  3. 2018Image Super-Resolution Using Very Deep Residual Channel Attention Networkscited by 1,256
  4. 2016Real-Time Single Image and Video Super-Resolution Using an Efficient Sub-Pixel Convolutional Neural Networkcited by 1,199
  5. 2021Video Swin Transformercited by 1,090
  6. 2015FlowNet: Learning Optical Flow with Convolutional Networkscited by 1,054
  7. 2017PWC-Net: CNNs for Optical Flow Using Pyramid, Warping, and Cost Volumecited by 966
  8. 2020Pre-Trained Image Processing Transformercited by 891
  9. 2021Uformer: A General U-Shaped Transformer for Image Restorationcited by 864
  10. 2021Multi-Stage Progressive Image Restorationcited by 813
  11. 2018Residual Dense Network for Image Super-Resolutioncited by 796
  12. 2016Deep Multi-scale Convolutional Neural Network for Dynamic Scene Deblurringcited by 722
  13. 2017Video Enhancement with Task-Oriented Flowcited by 612
  14. 2019Deep Learning for 3D Point Clouds: A Surveycited by 557
  15. 2021Designing a Practical Degradation Model for Deep Blind Image Super-Resolutioncited by 524
  16. 2019EDVR: Video Restoration with Enhanced Deformable Convolutional Networkscited by 492
  17. 2020Plug-and-Play Image Restoration with Deep Denoiser Priorcited by 467
  18. 2016Optical Flow Estimation using a Spatial Pyramid Networkcited by 421
  19. 2018Deep Learning for Generic Object Detection: A Surveycited by 392
  20. 2021BasicVSR++: Improving Video Super-Resolution with Enhanced Propagation and Alignmentcited by 340
  21. 2018Scale-recurrent Network for Deep Image Deblurringcited by 332
  22. 2017Super SloMo: High Quality Estimation of Multiple Intermediate Frames for Video Interpolationcited by 321
  23. 2020BasicVSR: The Search for Essential Components in Video Super-Resolution and Beyondcited by 303
  24. 2021Video Transformer Networkcited by 271
  25. 2019Residual Non-local Attention Networks for Image Restorationcited by 236
  26. 2019Depth-Aware Video Frame Interpolationcited by 232
  27. 2021LocalViT: Analyzing Locality in Vision Transformerscited by 228
  28. 2017Learning a Single Convolutional Super-Resolution Network for Multiple Degradationscited by 227
  29. 2018TDAN: Temporally Deformable Alignment Network for Video Super-Resolutioncited by 225
  30. 2017Video Frame Interpolation via Adaptive Separable Convolutioncited by 211
  31. 2017Video Frame Synthesis using Deep Voxel Flowcited by 209
  32. 2019Deep Stacked Hierarchical Multi-patch Network for Image Deblurringcited by 205
  33. 2021Unsupervised Degradation Representation Learning for Blind Super-Resolutioncited by 177
  34. 2018Video Object Segmentation with Language Referring Expressionscited by 161
  35. 2018Frame-Recurrent Video Super-Resolutioncited by 160
  36. 2018Non-Local Recurrent Network for Image Restorationcited by 160
  37. 2016Real-Time Video Super-Resolution with Spatio-Temporal Networks and Motion Compensationcited by 156
  38. 2017Detail-revealing Deep Video Super-resolutioncited by 147

50 most cited of 94 All 94 (JSON)

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2022: 35; 2023: 40; 2024: 51; 2025: 53; 2026: 40.

219 papers.

Sort:

By area:

  1. 2022Simple Baselines for Image Restorationcited by 701
  2. 2022Activating More Pixels in Image Super-Resolution Transformercited by 433
  3. 2022Recurrent Video Restoration Transformer with Guided Deformable Attentioncited by 168
  4. 2023Upscale-A-Video: Temporal-Consistent Diffusion Model for Real-World Video Super-Resolutioncited by 131
  5. 2022NAFSSR: Stereo Image Super-Resolution Using NAFNetcited by 75
  6. 2022Rethinking Alignment in Video Super-Resolution Transformerscited by 71
  7. 2022CiaoSR: Continuous Implicit Attention-in-Attention Network for Arbitrary-Scale Image Super-Resolutioncited by 69
  8. 2022A Simple Baseline for Video Restoration with Grouped Spatial-temporal Shiftcited by 68
  9. 2023HAT: Hybrid Attention Transformer for Image Restorationcited by 64
  10. 2024VEnhancer: Generative Space-Time Enhancement for Video Generationcited by 64
  11. 2025SeedVR: Seeding Infinity in Diffusion Transformer Towards Generic Video Restorationcited by 62
  12. 2023Motion-Guided Latent Diffusion for Temporally Consistent Real-world Video Super-resolutioncited by 59
38 more
  1. 2024A Survey on All-in-One Image Restoration: Taxonomy, Evaluation and Future Trendscited by 58
  2. 2025STAR: Spatial-Temporal Augmentation with Text-to-Video Models for Real-World Video Super-Resolutioncited by 56
  3. 2022N-Gram in Swin Transformers for Efficient Lightweight Image Super-Resolutioncited by 46
  4. 2025SeedVR2: One-Step Video Restoration via Diffusion Adversarial Post-Trainingcited by 45
  5. 2023DeSRA: Detect and Delete the Artifacts of GAN-based Real-World Super-Resolution Modelscited by 41
  6. 2023Enhancing Video Super-Resolution via Implicit Resampling-based Alignmentcited by 35
  7. 2024DiffIR2VR-Zero: Zero-Shot Video Restoration with Diffusion-based Image Restoration Modelscited by 33
  8. 2024RealViformer: Investigating Attention for Real-World Video Super-Resolutioncited by 33
  9. 2022BSRT: Improving Burst Super-Resolution with Swin Transformer and Flow-Guided Deformable Alignmentcited by 32
  10. 2022Imaging through the Atmosphere using Turbulence Mitigation Transformercited by 31
  11. 2024VideoGigaGAN: Towards Detail-rich Video Super-Resolutioncited by 30
  12. 2025DOVE: Efficient One-Step Diffusion Model for Real-World Video Super-Resolutioncited by 30
  13. 2022Mitigating Artifacts in Real-World Video Super-Resolution Modelscited by 29
  14. 2023NeRFLiX: High-Quality Neural View Synthesis by Learning a Degradation-Driven Inter-viewpoint MiXercited by 29
  15. 2024Video Super-Resolution Transformer with Masked Inter&Intra-Frame Attentioncited by 29
  16. 2024FMA-Net: Flow-Guided Dynamic Filtering and Iterative Feature Refinement with Multi-Attention for Joint Video Super-Resolution and Deblurringcited by 28
  17. 2023Skip-Attention: Improving Vision Transformers by Paying Less Attentioncited by 23
  18. 2024Learning Spatial Adaptation and Temporal Coherence in Diffusion Models for Video Super-Resolutioncited by 23
  19. 2022Enriched CNN-Transformer Feature Aggregation Networks for Super-Resolutioncited by 22
  20. 2024Spatio-Temporal Turbulence Mitigation: A Translational Perspectivecited by 22
  21. 2022Recent Advances in Vision Transformer: A Survey and Outlook of Recent Workcited by 21
  22. 2022AIM 2022 Challenge on Super-Resolution of Compressed Image and Video: Dataset, Methods and Resultscited by 20
  23. 2022On the Generalization of BasicVSR++ to Video Deblurring and Denoisingcited by 19
  24. 2023Enhancing Perceptual Quality in Video Super-Resolution through Temporally-Consistent Detail Synthesis using Diffusion Modelscited by 18
  25. 2023Deep Equilibrium Diffusion Restoration with Parallel Samplingcited by 17
  26. 2024Blur-aware Spatio-temporal Sparse Transformer for Video Deblurringcited by 17
  27. 2024HIIF: Hierarchical Encoding based Implicit Image Function for Continuous Super-resolutioncited by 17
  28. 2022Consistent Direct Time-of-Flight Video Depth Super-Resolutioncited by 16
  29. 2023Precipitation Downscaling with Spatiotemporal Video Diffusioncited by 16
  30. 2023MoVideo: Motion-Aware Video Generation with Diffusion Modelscited by 15
  31. 2026NTIRE 2026 Challenge on Bitstream-Corrupted Video Restoration: Methods and Resultscited by 15
  32. 2025AIM 2025 Low-light RAW Video Denoising Challenge: Dataset, Methods and Resultscited by 14
  33. 2022Video-based Human-Object Interaction Detection from Tubelet Tokenscited by 13
  34. 2024Learning Truncated Causal History Model for Video Restorationcited by 13
  35. 2026LUVE : Latent-Cascaded Ultra-High-Resolution Video Generation with Dual Frequency Expertscited by 13
  36. 2023Recaptured Raw Screen Image and Video Demoir'eing via Channel and Spatial Modulationscited by 12
  37. 2025MoA-VR: A Mixture-of-Agents System Towards All-in-One Video Restorationcited by 12
  38. 2022Unsupervised Flow-Aligned Sequence-to-Sequence Learning for Video Restorationcited by 11

50 most cited of 219 All 219 (JSON)