# VQFR: Blind Face Restoration with Vector-Quantized Dictionary and Parallel Decoder

arXiv: https://arxiv.org/abs/2205.06803 · HTML: https://mlsanity.com/paper/2205.06803/

## Key facts

- Cited by 116 papers in the map (data through 14 Oct 2026).
- Builds on 40 of its 48 references that are in the map.
- First posted on arXiv 13 May 2022.

## Builds on (most cited first, 30 of 40)

- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [Very Deep Convolutional Networks for Large-Scale Image Recognition](https://mlsanity.com/paper/1409.1556/) (2014)
- [The Unreasonable Effectiveness of Deep Features as a Perceptual Metric](https://mlsanity.com/paper/1801.03924/) (2018)
- [A Style-Based Generator Architecture for Generative Adversarial Networks](https://mlsanity.com/paper/1812.04948/) (2018)
- [Neural Discrete Representation Learning](https://mlsanity.com/paper/1711.00937/) (2017)
- [Image-to-Image Translation with Conditional Adversarial Networks](https://mlsanity.com/paper/1611.07004/) (2016)
- [Analyzing and Improving the Image Quality of StyleGAN](https://mlsanity.com/paper/1912.04958/) (2019)
- [Taming Transformers for High-Resolution Image Synthesis](https://mlsanity.com/paper/2012.09841/) (2020)
- [Perceptual Losses for Real-Time Style Transfer and Super-Resolution](https://mlsanity.com/paper/1603.08155/) (2016)
- [BEiT: BERT Pre-Training of Image Transformers](https://mlsanity.com/paper/2106.08254/) (2021)
- [Estimating or Propagating Gradients Through Stochastic Neurons for Conditional Computation](https://mlsanity.com/paper/1308.3432/) (2013)
- [Enhanced Deep Residual Networks for Single Image Super-Resolution](https://mlsanity.com/paper/1707.02921/) (2017)
- [Group Normalization](https://mlsanity.com/paper/1803.08494/) (2018)
- [Generating Diverse High-Fidelity Images with VQ-VAE-2](https://mlsanity.com/paper/1906.00446/) (2019)
- [Beyond a Gaussian Denoiser: Residual Learning of Deep CNN for Image Denoising](https://mlsanity.com/paper/1608.03981/) (2016)
- [MaskGIT: Masked Generative Image Transformer](https://mlsanity.com/paper/2202.04200/) (2022)
- [VGGFace2: A dataset for recognising faces across pose and age](https://mlsanity.com/paper/1710.08092/) (2017)
- [Deformable ConvNets v2: More Deformable, Better Results](https://mlsanity.com/paper/1811.11168/) (2018)
- [Vector-quantized Image Modeling with Improved VQGAN](https://mlsanity.com/paper/2110.04627/) (2021)
- [The Perception-Distortion Tradeoff](https://mlsanity.com/paper/1711.06077/) (2017)
- [DeblurGAN: Blind Motion Deblurring Using Conditional Adversarial Networks](https://mlsanity.com/paper/1711.07064/) (2017)
- [PULSE: Self-Supervised Photo Upsampling via Latent Space Exploration of Generative Models](https://mlsanity.com/paper/2003.03808/) (2020)
- [Towards Real-World Blind Face Restoration with Generative Facial Prior](https://mlsanity.com/paper/2101.04061/) (2021)
- [N"UWA: Visual Synthesis Pre-training for Neural visUal World creAtion](https://mlsanity.com/paper/2111.12417/) (2021)
- [PeCo: Perceptual Codebook for BERT Pre-training of Vision Transformers](https://mlsanity.com/paper/2111.12710/) (2021)
- [GAN Prior Embedded Network for Blind Face Restoration in the Wild](https://mlsanity.com/paper/2105.06070/) (2021)
- [Image Processing Using Multi-Code GAN Prior](https://mlsanity.com/paper/1912.07116/) (2019)
- [FSRNet: End-to-End Learning Face Super-Resolution with Facial Priors](https://mlsanity.com/paper/1711.10703/) (2017)
- [Adaptive Wing Loss for Robust Face Alignment via Heatmap Regression](https://mlsanity.com/paper/1904.07399/) (2019)

## Built on by (most cited first, 30 of 116)

- [DiffBIR: Towards Blind Image Restoration with Generative Diffusion Prior](https://mlsanity.com/paper/2308.15070/) (2023)
- [DifFace: Blind Face Restoration with Diffused Error Contraction](https://mlsanity.com/paper/2212.06512/) (2022)
- [DR2: Diffusion-based Robust Degradation Remover for Blind Face Restoration](https://mlsanity.com/paper/2303.06885/) (2023)
- [RIDCP: Revitalizing Real Image Dehazing via High-Quality Codebook Priors](https://mlsanity.com/paper/2304.03994/) (2023)
- [Efficient Diffusion Model for Image Restoration by Residual Shifting](https://mlsanity.com/paper/2403.07319/) (2024)
- [PGDiff: Guiding Diffusion Models for Versatile Face Restoration via Partial Guidance](https://mlsanity.com/paper/2309.10810/) (2023)
- [Diff-Plugin: Revitalizing Details for Diffusion-based Low-level Tasks](https://mlsanity.com/paper/2403.00644/) (2024)
- [Learning Dual Memory Dictionaries for Blind Face Restoration](https://mlsanity.com/paper/2210.08160/) (2022)
- [FlowVQTalker: High-Quality Emotional Talking Face Generation through Normalizing Flow and Quantization](https://mlsanity.com/paper/2403.06375/) (2024)
- [Posterior-Mean Rectified Flow: Towards Minimum MSE Photo-Realistic Image Restoration](https://mlsanity.com/paper/2410.00418/) (2024)
- [RestoreFormer++: Towards Real-World Blind Face Restoration from Undegraded Key-Value Pairs](https://mlsanity.com/paper/2308.07228/) (2023)
- [GLARE: Low Light Image Enhancement via Generative Latent Feature based Codebook Retrieval](https://mlsanity.com/paper/2407.12431/) (2024)
- [OSDFace: One-Step Diffusion Model for Face Restoration](https://mlsanity.com/paper/2411.17163/) (2024)
- [Rethinking the Objectives of Vector-Quantized Tokenizers for Image Synthesis](https://mlsanity.com/paper/2212.03185/) (2022)
- [Learned Image Compression with Dictionary-based Entropy Model](https://mlsanity.com/paper/2504.00496/) (2025)
- [Face Generation and Editing with StyleGAN: A Survey](https://mlsanity.com/paper/2212.09102/) (2022)
- [Compressed Image Generation with Denoising Diffusion Codebook Models](https://mlsanity.com/paper/2502.01189/) (2025)
- [SelfPromer: Self-Prompt Dehazing Transformers with Depth-Consistency](https://mlsanity.com/paper/2303.07033/) (2023)
- [Learning Generative Structure Prior for Blind Text Image Super-resolution](https://mlsanity.com/paper/2303.14726/) (2023)
- [Dual Associated Encoder for Face Restoration](https://mlsanity.com/paper/2308.07314/) (2023)
- [Towards Real-World Blind Face Restoration with Generative Diffusion Prior](https://mlsanity.com/paper/2312.15736/) (2023)
- [Effective Diffusion Transformer Architecture for Image Super-Resolution](https://mlsanity.com/paper/2409.19589/) (2024)
- [WaveFace: Authentic Face Restoration with Efficient Frequency Recovery](https://mlsanity.com/paper/2403.12760/) (2024)
- [Towards Authentic Face Restoration with Iterative Diffusion Models and Beyond](https://mlsanity.com/paper/2307.08996/) (2023)
- [Survey on Deep Face Restoration: From Non-blind to Blind and Beyond](https://mlsanity.com/paper/2309.15490/) (2023)
- [Deep Face Restoration: A Survey](https://mlsanity.com/paper/2211.02831/) (2022)
- [DiffBFR: Bootstrapping Diffusion Model Towards Blind Face Restoration](https://mlsanity.com/paper/2305.04517/) (2023)
- [Iterative Predictor-Critic Code Decoding for Real-World Image Dehazing](https://mlsanity.com/paper/2503.13147/) (2025)
- [Unlimited-Size Diffusion Restoration](https://mlsanity.com/paper/2303.00354/) (2023)
- [Robust Unsupervised StyleGAN Image Restoration](https://mlsanity.com/paper/2302.06733/) (2023)

## Abstract (arXiv)

Although generative facial prior and geometric prior have recently demonstrated high-quality results for blind face restoration, producing fine-grained facial details faithful to inputs remains a challenging problem. Motivated by the classical dictionary-based methods and the recent vector quantization (VQ) technique, we propose a VQ-based face restoration method - VQFR. VQFR takes advantage of high-quality low-level feature banks extracted from high-quality faces and can thus help recover realistic facial details. However, the simple application of the VQ codebook cannot achieve good results with faithful details and identity preservation. Therefore, we further introduce two special network designs. 1). We first investigate the compression patch size in the VQ codebook and find that the VQ codebook designed with a proper compression patch size is crucial to balance the quality and fidelity. 2). To further fuse low-level features from inputs while not "contaminating" the realistic details generated from the VQ codebook, we proposed a parallel decoder consisting of a texture decoder and a main decoder. Those two decoders then interact with a texture warping module with deformable convolution. Equipped with the VQ codebook as a facial detail dictionary and the parallel decoder design, the proposed VQFR can largely enhance the restored quality of facial details while keeping the fidelity to previous methods.