# Progressive Semantic-Aware Style Transformation for Blind Face Restoration

arXiv: https://arxiv.org/abs/2009.08709 · HTML: https://mlsanity.com/paper/2009.08709/

## Key facts

- Cited by 99 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 18 Sep 2020.

## Built on by (most cited first, 30 of 99)

- [Towards Real-World Blind Face Restoration with Generative Facial Prior](https://mlsanity.com/paper/2101.04061/) (2021)
- [DiffBIR: Towards Blind Image Restoration with Generative Diffusion Prior](https://mlsanity.com/paper/2308.15070/) (2023)
- [Towards Robust Blind Face Restoration with Codebook Lookup Transformer](https://mlsanity.com/paper/2206.11253/) (2022)
- [High-Fidelity GAN Inversion for Image Attribute Editing](https://mlsanity.com/paper/2109.06590/) (2021)
- [VFHQ: A High-Quality Dataset and Benchmark for Video Face Super-Resolution](https://mlsanity.com/paper/2205.03409/) (2022)
- [VQFR: Blind Face Restoration with Vector-Quantized Dictionary and Parallel Decoder](https://mlsanity.com/paper/2205.06803/) (2022)
- [Real-World Blind Super-Resolution via Feature Matching with Implicit High-Resolution Priors](https://mlsanity.com/paper/2202.13142/) (2022)
- [DifFace: Blind Face Restoration with Diffused Error Contraction](https://mlsanity.com/paper/2212.06512/) (2022)
- [RestoreFormer: High-Quality Blind Face Restoration from Undegraded Key-Value Pairs](https://mlsanity.com/paper/2201.06374/) (2022)
- [DR2: Diffusion-based Robust Degradation Remover for Blind Face Restoration](https://mlsanity.com/paper/2303.06885/) (2023)
- [Efficient Diffusion Model for Image Restoration by Residual Shifting](https://mlsanity.com/paper/2403.07319/) (2024)
- [Learning Dual Memory Dictionaries for Blind Face Restoration](https://mlsanity.com/paper/2210.08160/) (2022)
- [GCFSR: a Generative and Controllable Face Super Resolution Method Without Facial and GAN Priors](https://mlsanity.com/paper/2203.07319/) (2022)
- [RestoreFormer++: Towards Real-World Blind Face Restoration from Undegraded Key-Value Pairs](https://mlsanity.com/paper/2308.07228/) (2023)
- [OSDFace: One-Step Diffusion Model for Face Restoration](https://mlsanity.com/paper/2411.17163/) (2024)
- [Deep Learning-based Face Super-Resolution: A Survey](https://mlsanity.com/paper/2101.03749/) (2021)
- [IFQA: Interpretable Face Quality Assessment](https://mlsanity.com/paper/2211.07077/) (2022)
- [Dual Associated Encoder for Face Restoration](https://mlsanity.com/paper/2308.07314/) (2023)
- [Towards Real-World Blind Face Restoration with Generative Diffusion Prior](https://mlsanity.com/paper/2312.15736/) (2023)
- [LTT-GAN: Looking Through Turbulence by Inverting GANs](https://mlsanity.com/paper/2112.02379/) (2021)
- [Effective Diffusion Transformer Architecture for Image Super-Resolution](https://mlsanity.com/paper/2409.19589/) (2024)
- [WaveFace: Authentic Face Restoration with Efficient Frequency Recovery](https://mlsanity.com/paper/2403.12760/) (2024)
- [GLEAN: Generative Latent Bank for Image Super-Resolution and Beyond](https://mlsanity.com/paper/2207.14812/) (2022)
- [Towards Authentic Face Restoration with Iterative Diffusion Models and Beyond](https://mlsanity.com/paper/2307.08996/) (2023)
- [Survey on Deep Face Restoration: From Non-blind to Blind and Beyond](https://mlsanity.com/paper/2309.15490/) (2023)
- [Deep Face Restoration: A Survey](https://mlsanity.com/paper/2211.02831/) (2022)
- [DiffBFR: Bootstrapping Diffusion Model Towards Blind Face Restoration](https://mlsanity.com/paper/2305.04517/) (2023)
- [From Face to Natural Image: Learning Real Degradation for Blind Image Super-Resolution](https://mlsanity.com/paper/2210.00752/) (2022)
- [PFStorer: Personalized Face Restoration and Super-Resolution](https://mlsanity.com/paper/2403.08436/) (2024)
- [InstantRestore: Single-Step Personalized Face Restoration with Shared-Image Attention](https://mlsanity.com/paper/2412.06753/) (2024)

## Abstract (arXiv)

Face restoration is important in face image processing, and has been widely studied in recent years. However, previous works often fail to generate plausible high quality (HQ) results for real-world low quality (LQ) face images. In this paper, we propose a new progressive semantic-aware style transformation framework, named PSFR-GAN, for face restoration. Specifically, instead of using an encoder-decoder framework as previous methods, we formulate the restoration of LQ face images as a multi-scale progressive restoration procedure through semantic-aware style transformation. Given a pair of LQ face image and its corresponding parsing map, we first generate a multi-scale pyramid of the inputs, and then progressively modulate different scale features from coarse-to-fine in a semantic-aware style transfer way. Compared with previous networks, the proposed PSFR-GAN makes full use of the semantic (parsing maps) and pixel (LQ images) space information from different scales of input pairs. In addition, we further introduce a semantic aware style loss which calculates the feature style loss for each semantic region individually to improve the details of face textures. Finally, we pretrain a face parsing network which can generate decent parsing maps from real-world LQ face images. Experiment results show that our model trained with synthetic data can not only produce more realistic high-resolution results for synthetic LQ inputs and but also generalize better to natural LQ face images compared with state-of-the-art methods. Codes are available at https://github.com/chaofengc/PSFRGAN.