Analysis and Benchmarking of Extending Blind Face Image Restoration to Videos
Not on the map. Why?
Abstract (arXiv)
Recent progress in blind face restoration has resulted in producing high-quality restored results for static images. However, efforts to extend these advancements to video scenarios have been minimal, partly because of the absence of benchmarks that allow for a comprehensive and fair comparison. In this work, we first present a fair evaluation benchmark, in which we first introduce a Real-world Low-Quality Face Video benchmark (RFV-LQ), evaluate several leading image-based face restoration algorithms, and conduct a thorough systematical analysis of the benefits and challenges associated with extending blind face image restoration algorithms to degraded face videos. Our analysis identifies several key issues, primarily categorized into two aspects: significant jitters in facial components and noise-shape flickering between frames. To address these issues, we propose a Temporal Consistency Network (TCN) cooperated with alignment smoothing to reduce jitters and flickers in restored videos. TCN is a flexible component that can be seamlessly plugged into the most advanced face image restoration algorithms, ensuring the quality of image-based restoration is maintained as closely as possible. Extensive experiments have been conducted to evaluate the effectiveness and efficiency of our proposed TCN and alignment smoothing operation. Project page: https://wzhouxiff.github.io/projects/FIR2FVR/FIR2FVR.
Key facts. Cited by 2 papers in the map (data through 14 Oct 2026). Builds on 39 of its 53 references that are in the map. First posted on arXiv 15 Oct 2024.
Builds on
39 of 53 references.
References · 39
- 2015Deep Residual Learning for Image Recognitioncited by 52,096
- 2014Very Deep Convolutional Networks for Large-Scale Image Recognitioncited by 15,364
- 2018A Style-Based Generator Architecture for Generative Adversarial Networkscited by 6,013
- 2022Generative Adversarial Networkscited by 5,627
- 2017Neural Discrete Representation Learningcited by 4,712
- 2019Analyzing and Improving the Image Quality of StyleGANcited by 3,588
- 2016Simple Online and Realtime Trackingcited by 868
- 2019EDVR: Video Restoration with Enhanced Deformable Convolutional Networkscited by 494
- 2016Optical Flow Estimation using a Spatial Pyramid Networkcited by 424
- 2020PULSE: Self-Supervised Photo Upsampling via Latent Space Exploration of Generative Modelscited by 358
- 2021Towards Real-World Blind Face Restoration with Generative Facial Priorcited by 349
- 2020BasicVSR: The Search for Essential Components in Video Super-Resolution and Beyondcited by 304
27 more
- 2018Learning Blind Video Temporal Consistencycited by 253
- 2022Towards Robust Blind Face Restoration with Codebook Lookup Transformercited by 231
- 2021GAN Prior Embedded Network for Blind Face Restoration in the Wildcited by 197
- 2019Image Processing Using Multi-Code GAN Priorcited by 185
- 2022VFHQ: A High-Quality Dataset and Benchmark for Video Face Super-Resolutioncited by 176
- 2017FSRNet: End-to-End Learning Face Super-Resolution with Facial Priorscited by 170
- 2019Adaptive Wing Loss for Robust Face Alignment via Heatmap Regressioncited by 128
- 2022VQFR: Blind Face Restoration with Vector-Quantized Dictionary and Parallel Decodercited by 116
- 2020Blind Face Restoration via Deep Multi-scale Component Dictionariescited by 106
- 2020Progressive Semantic-Aware Style Transformation for Blind Face Restorationcited by 99
- 2020Bringing Old Photos Back to Lifecited by 95
- 2018Learning Warped Guidance for Blind Face Restorationcited by 87
- 2022RestoreFormer: High-Quality Blind Face Restoration from Undegraded Key-Value Pairscited by 85
- 2018Deep Semantic Face Deblurringcited by 80
- 2020Blind Video Temporal Consistency via Deep Video Priorcited by 76
- 2019Progressive Face Super-Resolution via Attention to Facial Landmarkcited by 73
- 2020HiFaceGAN: Face Renovation via Collaborative Suppression and Replenishmentcited by 70
- 2019Exemplar Guided Face Image Super-Resolution without Facial Landmarkscited by 57
- 2022Learning Dual Memory Dictionaries for Blind Face Restorationcited by 40
- 2016Deep Cascaded Bi-Network for Face Hallucinationcited by 37
- 2023RestoreFormer++: Towards Real-World Blind Face Restoration from Undegraded Key-Value Pairscited by 33
- 2022Temporally Consistent Semantic Video Editingcited by 28
- 2023Blind Video Deflickering by Neural Filtering with a Flawed Atlascited by 27
- 2019Deblurring Face Images using Uncertainty Guided Multi-Stream Semantic Networkscited by 24
- 2022CTCNet: A CNN-Transformer Cooperation Network for Face Image Super-Resolutioncited by 19
- 2022Deep Face Restoration: A Surveycited by 17
- 2022StableFace: Analyzing and Improving Motion Stability for Talking Face Generationcited by 9
Built on by
Citing papers per year: 2026: 2.
2 papers.