# Beyond Alignment: Blind Video Face Restoration via Parsing-Guided Temporal-Coherent Transformer

arXiv: https://arxiv.org/abs/2404.13640 · HTML: https://mlsanity.com/paper/2404.13640/

## Key facts

- Cited by 11 papers in the map (data through 14 Oct 2026).
- Builds on 30 of its 40 references that are in the map.
- First posted on arXiv 21 Apr 2024.

## Builds on (most cited first, 30 of 30)

- [The Unreasonable Effectiveness of Deep Features as a Perceptual Metric](https://mlsanity.com/paper/1801.03924/) (2018)
- [Video Swin Transformer](https://mlsanity.com/paper/2106.13230/) (2021)
- [MaskGAN: Towards Diverse and Interactive Facial Image Manipulation](https://mlsanity.com/paper/1907.11922/) (2019)
- [EDVR: Video Restoration with Enhanced Deformable Convolutional Networks](https://mlsanity.com/paper/1905.02716/) (2019)
- [PULSE: Self-Supervised Photo Upsampling via Latent Space Exploration of Generative Models](https://mlsanity.com/paper/2003.03808/) (2020)
- [Towards Real-World Blind Face Restoration with Generative Facial Prior](https://mlsanity.com/paper/2101.04061/) (2021)
- [BasicVSR++: Improving Video Super-Resolution with Enhanced Propagation and Alignment](https://mlsanity.com/paper/2104.13371/) (2021)
- [BasicVSR: The Search for Essential Components in Video Super-Resolution and Beyond](https://mlsanity.com/paper/2012.02181/) (2020)
- [BiSeNet V2: Bilateral Network with Guided Aggregation for Real-time Semantic Segmentation](https://mlsanity.com/paper/2004.02147/) (2020)
- [Towards Robust Blind Face Restoration with Codebook Lookup Transformer](https://mlsanity.com/paper/2206.11253/) (2022)
- [GAN Prior Embedded Network for Blind Face Restoration in the Wild](https://mlsanity.com/paper/2105.06070/) (2021)
- [Image Processing Using Multi-Code GAN Prior](https://mlsanity.com/paper/1912.07116/) (2019)
- [VFHQ: A High-Quality Dataset and Benchmark for Video Face Super-Resolution](https://mlsanity.com/paper/2205.03409/) (2022)
- [FSRNet: End-to-End Learning Face Super-Resolution with Facial Priors](https://mlsanity.com/paper/1711.10703/) (2017)
- [GLEAN: Generative Latent Bank for Large-Factor Image Super-Resolution](https://mlsanity.com/paper/2012.00739/) (2020)
- [Adaptive Wing Loss for Robust Face Alignment via Heatmap Regression](https://mlsanity.com/paper/1904.07399/) (2019)
- [VQFR: Blind Face Restoration with Vector-Quantized Dictionary and Parallel Decoder](https://mlsanity.com/paper/2205.06803/) (2022)
- [Progressive Semantic-Aware Style Transformation for Blind Face Restoration](https://mlsanity.com/paper/2009.08709/) (2020)
- [ArcFace: Additive Angular Margin Loss for Deep Face Recognition](https://mlsanity.com/paper/1801.07698/) (2018)
- [MFQE 2.0: A New Approach for Multi-frame Quality Enhancement on Compressed Video](https://mlsanity.com/paper/1902.09707/) (2019)
- [Multi-Frame Quality Enhancement for Compressed Video](https://mlsanity.com/paper/1803.04680/) (2018)
- [SuperYOLO: Super Resolution Assisted Object Detection in Multimodal Remote Sensing Imagery](https://mlsanity.com/paper/2209.13351/) (2022)
- [Video Demoireing with Relation-Based Temporal Consistency](https://mlsanity.com/paper/2204.02957/) (2022)
- [SelfPromer: Self-Prompt Dehazing Transformers with Depth-Consistency](https://mlsanity.com/paper/2303.07033/) (2023)
- [SDRTV-to-HDRTV via Hierarchical Dynamic Context Feature Mapping](https://mlsanity.com/paper/2207.00319/) (2022)
- [Guided Hybrid Quantization for Object detection in Multimodal Remote Sensing Imagery via One-to-one Self-teaching](https://mlsanity.com/paper/2301.00131/) (2022)
- [Transcoded Video Restoration by Temporal Spatial Auxiliary Network](https://mlsanity.com/paper/2112.07948/) (2021)
- [SDRTV-to-HDRTV Conversion via Spatial-Temporal Feature Fusion](https://mlsanity.com/paper/2211.02297/) (2022)
- [A Hierarchical Interactive Network for Joint Span-based Aspect-Sentiment Analysis](https://mlsanity.com/paper/2208.11283/) (2022)
- [Dual Inverse Degradation Network for Real-World SDRTV-to-HDRTV Conversion](https://mlsanity.com/paper/2307.03394/) (2023)

## Built on by (most cited first, 11 of 11)

- [Smoothness Really Matters: A Simple Yet Effective Approach for Unsupervised Graph Domain Adaptation](https://mlsanity.com/paper/2412.11654/) (2024)
- [Intra and Inter Parser-Prompted Transformers for Effective Image Restoration](https://mlsanity.com/paper/2503.14037/) (2025)
- [Efficient Video Face Enhancement with Enhanced Spatial-Temporal Consistency](https://mlsanity.com/paper/2411.16468/) (2024)
- [DicFace: Dirichlet-Constrained Variational Codebook Learning for Temporally Coherent Video Face Restoration](https://mlsanity.com/paper/2506.13355/) (2025)
- [RealRep: Generalized SDR-to-HDR Conversion via Attribute-Disentangled Representation Learning](https://mlsanity.com/paper/2505.07322/) (2025)
- [Disentangling Granularity: An Implicit Inductive Bias in Factorized VAEs](https://mlsanity.com/paper/2505.24684/) (2025)
- [DVFace: Spatio-Temporal Dual-Prior Diffusion for Video Face Restoration](https://mlsanity.com/paper/2604.14560/) (2026)
- [RGFVR: Reference-Guided Face Video Restoration with Flow Matching](https://mlsanity.com/paper/2606.16401/) (2026)
- [TIGER: Taming Identity, Geometry, and Generative Priors for High-Quality Face Video Restoration](https://mlsanity.com/paper/2606.24336/) (2026)
- [DTI: Dynamic Trajectory Initialization for Generative Face Video Super-Resolution](https://mlsanity.com/paper/2606.29198/) (2026)
- [FADRA: Frequency-Aware Diffusion with Residual Adaptation for Video Face Restoration](https://mlsanity.com/paper/2607.06389/) (2026)

## Abstract (arXiv)

Multiple complex degradations are coupled in low-quality video faces in the real world. Therefore, blind video face restoration is a highly challenging ill-posed problem, requiring not only hallucinating high-fidelity details but also enhancing temporal coherence across diverse pose variations. Restoring each frame independently in a naive manner inevitably introduces temporal incoherence and artifacts from pose changes and keypoint localization errors. To address this, we propose the first blind video face restoration approach with a novel parsing-guided temporal-coherent transformer (PGTFormer) without pre-alignment. PGTFormer leverages semantic parsing guidance to select optimal face priors for generating temporally coherent artifact-free results. Specifically, we pre-train a temporal-spatial vector quantized auto-encoder on high-quality video face datasets to extract expressive context-rich priors. Then, the temporal parse-guided codebook predictor (TPCP) restores faces in different poses based on face parsing context cues without performing face pre-alignment. This strategy reduces artifacts and mitigates jitter caused by cumulative errors from face pre-alignment. Finally, the temporal fidelity regulator (TFR) enhances fidelity through temporal feature interaction and improves video temporal consistency. Extensive experiments on face videos show that our method outperforms previous face restoration baselines. The code will be released on hrefhttps://github.com/kepengxu/PGTFormerhttps://github.com/kepengxu/PGTFormer.