# Towards Robust Blind Face Restoration with Codebook Lookup Transformer

arXiv: https://arxiv.org/abs/2206.11253 · HTML: https://mlsanity.com/paper/2206.11253/

## Key facts

- Cited by 231 papers in the map (data through 14 Oct 2026).
- Builds on 33 of its 62 references that are in the map.
- First posted on arXiv 22 Jun 2022.

## Builds on (most cited first, 30 of 33)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [High-Resolution Image Synthesis with Latent Diffusion Models](https://mlsanity.com/paper/2112.10752/) (2021)
- [Very Deep Convolutional Networks for Large-Scale Image Recognition](https://mlsanity.com/paper/1409.1556/) (2014)
- [The Unreasonable Effectiveness of Deep Features as a Perceptual Metric](https://mlsanity.com/paper/1801.03924/) (2018)
- [End-to-End Object Detection with Transformers](https://mlsanity.com/paper/2005.12872/) (2020)
- [A Style-Based Generator Architecture for Generative Adversarial Networks](https://mlsanity.com/paper/1812.04948/) (2018)
- [Neural Discrete Representation Learning](https://mlsanity.com/paper/1711.00937/) (2017)
- [Taming Transformers for High-Resolution Image Synthesis](https://mlsanity.com/paper/2012.09841/) (2020)
- [Progressive Growing of GANs for Improved Quality, Stability, and Variation](https://mlsanity.com/paper/1710.10196/) (2017)
- [Perceptual Losses for Real-Time Style Transfer and Super-Resolution](https://mlsanity.com/paper/1603.08155/) (2016)
- [Masked-attention Mask Transformer for Universal Image Segmentation](https://mlsanity.com/paper/2112.01527/) (2021)
- [MaskGIT: Masked Generative Image Transformer](https://mlsanity.com/paper/2202.04200/) (2022)
- [MUSIQ: Multi-scale Image Quality Transformer](https://mlsanity.com/paper/2108.05997/) (2021)
- [Encoding in Style: a StyleGAN Encoder for Image-to-Image Translation](https://mlsanity.com/paper/2008.00951/) (2020)
- [Recovering Realistic Texture in Image Super-resolution by Deep Spatial Feature Transform](https://mlsanity.com/paper/1804.02815/) (2018)
- [PULSE: Self-Supervised Photo Upsampling via Latent Space Exploration of Generative Models](https://mlsanity.com/paper/2003.03808/) (2020)
- [Towards Real-World Blind Face Restoration with Generative Facial Prior](https://mlsanity.com/paper/2101.04061/) (2021)
- [WIDER FACE: A Face Detection Benchmark](https://mlsanity.com/paper/1511.06523/) (2015)
- [GAN Prior Embedded Network for Blind Face Restoration in the Wild](https://mlsanity.com/paper/2105.06070/) (2021)
- [Image Processing Using Multi-Code GAN Prior](https://mlsanity.com/paper/1912.07116/) (2019)
- [FSRNet: End-to-End Learning Face Super-Resolution with Facial Priors](https://mlsanity.com/paper/1711.10703/) (2017)
- [GLEAN: Generative Latent Bank for Large-Factor Image Super-Resolution](https://mlsanity.com/paper/2012.00739/) (2020)
- [Blind Face Restoration via Deep Multi-scale Component Dictionaries](https://mlsanity.com/paper/2008.00418/) (2020)
- [Progressive Semantic-Aware Style Transformation for Blind Face Restoration](https://mlsanity.com/paper/2009.08709/) (2020)
- [Cross-Scale Internal Graph Neural Network for Image Super-Resolution](https://mlsanity.com/paper/2006.16673/) (2020)
- [Learning Warped Guidance for Blind Face Restoration](https://mlsanity.com/paper/1804.04829/) (2018)
- [RestoreFormer: High-Quality Blind Face Restoration from Undegraded Key-Value Pairs](https://mlsanity.com/paper/2201.06374/) (2022)
- [Deep Semantic Face Deblurring](https://mlsanity.com/paper/1803.03345/) (2018)
- [HiFaceGAN: Face Renovation via Collaborative Suppression and Replenishment](https://mlsanity.com/paper/2005.05005/) (2020)

## Built on by (most cited first, 30 of 231)

- [Exploiting Diffusion Prior for Real-World Image Super-Resolution](https://mlsanity.com/paper/2305.07015/) (2023)
- [LAVIE: High-Quality Video Generation with Cascaded Latent Diffusion Models](https://mlsanity.com/paper/2309.15103/) (2023)
- [Scaling Up to Excellence: Practicing Model Scaling for Photo-Realistic Image Restoration In the Wild](https://mlsanity.com/paper/2401.13627/) (2024)
- [DiffBIR: Towards Blind Image Restoration with Generative Diffusion Prior](https://mlsanity.com/paper/2308.15070/) (2023)
- [Pixel-Aware Stable Diffusion for Realistic Image Super-resolution and Personalized Stylization](https://mlsanity.com/paper/2308.14469/) (2023)
- [CodeTalker: Speech-Driven 3D Facial Animation with Discrete Motion Prior](https://mlsanity.com/paper/2301.02379/) (2023)
- [AutoDIR: Automatic All-in-One Image Restoration with Latent Diffusion](https://mlsanity.com/paper/2310.10123/) (2023)
- [DifFace: Blind Face Restoration with Diffused Error Contraction](https://mlsanity.com/paper/2212.06512/) (2022)
- [Hallo2: Long-Duration and High-Resolution Audio-Driven Portrait Image Animation](https://mlsanity.com/paper/2410.07718/) (2024)
- [StableAnimator: High-Quality Identity-Preserving Human Image Animation](https://mlsanity.com/paper/2411.17697/) (2024)
- [DR2: Diffusion-based Robust Degradation Remover for Blind Face Restoration](https://mlsanity.com/paper/2303.06885/) (2023)
- [RIDCP: Revitalizing Real Image Dehazing via High-Quality Codebook Priors](https://mlsanity.com/paper/2304.03994/) (2023)
- [Diffusion Models, Image Super-Resolution And Everything: A Survey](https://mlsanity.com/paper/2401.00736/) (2024)
- [Efficient Diffusion Model for Image Restoration by Residual Shifting](https://mlsanity.com/paper/2403.07319/) (2024)
- [SeedVR: Seeding Infinity in Diffusion Transformer Towards Generic Video Restoration](https://mlsanity.com/paper/2501.01320/) (2025)
- [Motion-Guided Latent Diffusion for Temporally Consistent Real-world Video Super-resolution](https://mlsanity.com/paper/2312.00853/) (2023)
- [SeedVR2: One-Step Video Restoration via Diffusion Adversarial Post-Training](https://mlsanity.com/paper/2506.05301/) (2025)
- [PGDiff: Guiding Diffusion Models for Versatile Face Restoration via Partial Guidance](https://mlsanity.com/paper/2309.10810/) (2023)
- [FlowVQTalker: High-Quality Emotional Talking Face Generation through Normalizing Flow and Quantization](https://mlsanity.com/paper/2403.06375/) (2024)
- [SAR3D: Autoregressive 3D Object Generation and Understanding via Multi-scale 3D VQVAE](https://mlsanity.com/paper/2411.16856/) (2024)
- [Posterior-Mean Rectified Flow: Towards Minimum MSE Photo-Realistic Image Restoration](https://mlsanity.com/paper/2410.00418/) (2024)
- [RestoreFormer++: Towards Real-World Blind Face Restoration from Undegraded Key-Value Pairs](https://mlsanity.com/paper/2308.07228/) (2023)
- [GLARE: Low Light Image Enhancement via Generative Latent Feature based Codebook Retrieval](https://mlsanity.com/paper/2407.12431/) (2024)
- [OSDFace: One-Step Diffusion Model for Face Restoration](https://mlsanity.com/paper/2411.17163/) (2024)
- [FaithDiff: Unleashing Diffusion Priors for Faithful Image Super-resolution](https://mlsanity.com/paper/2411.18824/) (2024)
- [Rethinking the Objectives of Vector-Quantized Tokenizers for Image Synthesis](https://mlsanity.com/paper/2212.03185/) (2022)
- [ARTalk: Speech-Driven 3D Head Animation via Autoregressive Model](https://mlsanity.com/paper/2502.20323/) (2025)
- [Face Generation and Editing with StyleGAN: A Survey](https://mlsanity.com/paper/2212.09102/) (2022)
- [Compressed Image Generation with Denoising Diffusion Codebook Models](https://mlsanity.com/paper/2502.01189/) (2025)
- [Zero-shot Generation of Coherent Storybook from Plain Text Story using Diffusion Models](https://mlsanity.com/paper/2302.03900/) (2023)

## Abstract (arXiv)

Blind face restoration is a highly ill-posed problem that often requires auxiliary guidance to 1) improve the mapping from degraded inputs to desired outputs, or 2) complement high-quality details lost in the inputs. In this paper, we demonstrate that a learned discrete codebook prior in a small proxy space largely reduces the uncertainty and ambiguity of restoration mapping by casting blind face restoration as a code prediction task, while providing rich visual atoms for generating high-quality faces. Under this paradigm, we propose a Transformer-based prediction network, named CodeFormer, to model the global composition and context of the low-quality faces for code prediction, enabling the discovery of natural faces that closely approximate the target faces even when the inputs are severely degraded. To enhance the adaptiveness for different degradation, we also propose a controllable feature transformation module that allows a flexible trade-off between fidelity and quality. Thanks to the expressive codebook prior and global modeling, CodeFormer outperforms the state of the arts in both quality and fidelity, showing superior robustness to degradation. Extensive experimental results on synthetic and real-world datasets verify the effectiveness of our method.