mlsanityData to 7 Oct 2026

Progressive Training of A Two-Stage Framework for Video Restoration

arXiv 2204.09924 ↗First posted 21 Apr 2022cs.CV cs.MMcited by 7

Not on the map. Why?

Abstract (arXiv)

As a widely studied task, video restoration aims to enhance the quality of the videos with multiple potential degradations, such as noises, blurs and compression artifacts. Among video restorations, compressed video quality enhancement and video super-resolution are two of the main tacks with significant values in practical scenarios. Recently, recurrent neural networks and transformers attract increasing research interests in this field, due to their impressive capability in sequence-to-sequence modeling. However, the training of these models is not only costly but also relatively hard to converge, with gradient exploding and vanishing problems. To cope with these problems, we proposed a two-stage framework including a multi-frame recurrent network and a single-frame transformer. Besides, multiple training strategies, such as transfer learning and progressive training, are developed to shorten the training time and improve the model performance. Benefiting from the above technical contributions, our solution wins two champions and a runner-up in the NTIRE 2022 super-resolution and quality enhancement of compressed video challenges. Code is available at https://github.com/ryanxingql/winner-ntire22-vqe.

Key facts. Cited by 7 papers in the map (data through 7 Oct 2026). Builds on 41 of its 45 references that are in the map. First posted on arXiv 21 Apr 2022.

Builds on

41 of 45 references.

References · 41

  1. 2015Deep Residual Learning for Image Recognitioncited by 51,809
  2. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,204
  3. 2020An Image is Worth 16x16 Words: Transformers for Image Recognition at Scalecited by 29,915
  4. 2019RoBERTa: A Robustly Optimized BERT Pretraining Approachcited by 13,266
  5. 2021Swin Transformer: Hierarchical Vision Transformer using Shifted Windowscited by 12,082
  6. 2020End-to-End Object Detection with Transformerscited by 7,384
  7. 2021Masked Autoencoders Are Scalable Vision Learnerscited by 7,339
  8. 2019ALBERT: A Lite BERT for Self-supervised Learning of Language Representationscited by 2,772
  9. 2021SwinIR: Image Restoration Using Swin Transformercited by 1,869
  10. 2021Restormer: Efficient Transformer for High-Resolution Image Restorationcited by 1,618
  11. 2021ViViT: A Video Vision Transformercited by 1,610
  12. 2017Deformable Convolutional Networkscited by 1,481
29 more
  1. 2016Real-Time Single Image and Video Super-Resolution Using an Efficient Sub-Pixel Convolutional Neural Networkcited by 1,199
  2. 2020Pre-Trained Image Processing Transformercited by 891
  3. 2021Uformer: A General U-Shaped Transformer for Image Restorationcited by 864
  4. 2018Deformable ConvNets v2: More Deformable, Better Resultscited by 666
  5. 2017Video Enhancement with Task-Oriented Flowcited by 612
  6. 2019EDVR: Video Restoration with Enhanced Deformable Convolutional Networkscited by 492
  7. 2016Optical Flow Estimation using a Spatial Pyramid Networkcited by 421
  8. 2020Learning Texture Transformer Network for Image Super-Resolutioncited by 346
  9. 2021BasicVSR++: Improving Video Super-Resolution with Enhanced Propagation and Alignmentcited by 340
  10. 2020BasicVSR: The Search for Essential Components in Video Super-Resolution and Beyondcited by 303
  11. 2022Vision Transformer with Deformable Attentioncited by 293
  12. 2022VRT: A Video Restoration Transformercited by 219
  13. 2021TransGAN: Two Pure Transformers Can Make One Strong GAN, and That Can Scale Upcited by 162
  14. 2021HRFormer: High-Resolution Transformer for Dense Predictioncited by 149
  15. 2015Compression Artifacts Reduction by a Deep Convolutional Networkcited by 136
  16. 2021Video Super-Resolution Transformercited by 120
  17. 2020Video Super-Resolution with Recurrent Structure-Detail Networkcited by 116
  18. 2020MuCAN: Multi-Correspondence Aggregation Network for Video Super-Resolutioncited by 89
  19. 2021Perceptual Image Quality Assessment with Transformerscited by 54
  20. 2015Seven ways to improve example-based single image super resolutioncited by 51
  21. 2019MFQE 2.0: A New Approach for Multi-frame Quality Enhancement on Compressed Videocited by 50
  22. 2018Multi-Frame Quality Enhancement for Compressed Videocited by 48
  23. 2020Deformable 3D Convolution for Video Super-Resolutioncited by 32
  24. 2020Early Exit or Not: Resource-Efficient Blind Quality Enhancement for Compressed Imagescited by 28
  25. 2021NTIRE 2021 Challenge on Quality Enhancement of Compressed Video: Methods and Resultscited by 26
  26. 2022NTIRE 2022 Challenge on Super-Resolution and Quality Enhancement of Compressed Video: Dataset, Methods and Resultscited by 20
  27. 2021NTIRE 2021 Challenge on Quality Enhancement of Compressed Video: Dataset and Studycited by 9
  28. 2020Progressive Training of Multi-level Wavelet Residual Networks for Image Denoisingcited by 3
  29. 2021PP-MSVSR: Multi-Stage Video Super-Resolutioncited by 2

Built on by

Cited by year

Citing papers per year: 2022: 3; 2023: 1; 2024: 1; 2025: 2.

7 papers.

  1. 2022HST: Hierarchical Swin Transformer for Compressed Image Super-resolutioncited by 20
  2. 2022AIM 2022 Challenge on Super-Resolution of Compressed Image and Video: Dataset, Methods and Resultscited by 20
  3. 2022DAQE: Enhancing the Quality of Compressed Images by Exploiting the Inherent Characteristic of Defocuscited by 2
  4. 2023Deep Video Restoration for Under-Display Cameracited by 2
  5. 2024Enhancing Quality of Compressed Images by Mitigating Enhancement Bias Towards Compression Domaincited by 1
  6. 2025Higher fidelity perceptual image and video compression with a latent conditioned residual denoising diffusion modelcited by 1
  7. 2025RED: Robust Event-Guided Motion Deblurring with Modality-Specific Disentanglementcited by 0