# DR-BFR: Degradation Representation with Diffusion Models for Blind Face Restoration

arXiv: https://arxiv.org/abs/2411.10508 · HTML: https://mlsanity.com/paper/2411.10508/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Placed in Inverse problems and image restoration, one of 14 directions in Diffusion models.
- Builds on 17 of its 55 references that are in the map.
- First posted on arXiv 15 Nov 2024.

## Placed in

- Diffusion models / [Inverse problems and image restoration](https://mlsanity.com/diffusion/inverse-problems-and-image-restoration/)

## Builds on (most cited first, 17 of 17)

- [Very Deep Convolutional Networks for Large-Scale Image Recognition](https://mlsanity.com/paper/1409.1556/) (2014)
- [Denoising Diffusion Implicit Models](https://mlsanity.com/paper/2010.02502/) (2020)
- [Conditional Generative Adversarial Nets](https://mlsanity.com/paper/1411.1784/) (2014)
- [DiffBIR: Towards Blind Image Restoration with Generative Diffusion Prior](https://mlsanity.com/paper/2308.15070/) (2023)
- [Inversion by Direct Iteration: An Alternative to Denoising Diffusion for Image Restoration](https://mlsanity.com/paper/2303.11435/) (2023)
- [AutoDIR: Automatic All-in-One Image Restoration with Latent Diffusion](https://mlsanity.com/paper/2310.10123/) (2023)
- [Efficient Diffusion Model for Image Restoration by Residual Shifting](https://mlsanity.com/paper/2403.07319/) (2024)
- [Neural Degradation Representation Learning for All-In-One Image Restoration](https://mlsanity.com/paper/2310.12848/) (2023)
- [Image Super-Resolution with Text Prompt Diffusion](https://mlsanity.com/paper/2311.14282/) (2023)
- [PromptRR: Diffusion Models as Prompt Generators for Single Image Reflection Removal](https://mlsanity.com/paper/2402.02374/) (2024)
- [Efficient Degradation-aware Any Image Restoration](https://mlsanity.com/paper/2405.15475/) (2024)
- [O2V-Mapping: Online Open-Vocabulary Mapping with Neural Implicit Representation](https://mlsanity.com/paper/2404.06836/) (2024)
- [UCIP: A Universal Framework for Compressed Image Super-Resolution using Dynamic Prompt](https://mlsanity.com/paper/2407.13108/) (2024)
- [The Dawn of KAN in Image-to-Image (I2I) Translation: Integrating Kolmogorov-Arnold Networks with GANs for Unpaired I2I Translation](https://mlsanity.com/paper/2408.08216/) (2024)
- [Seek for Incantations: Towards Accurate Text-to-Image Diffusion Synthesis through Prompt Engineering](https://mlsanity.com/paper/2401.06345/) (2024)
- [Preconditioners for the Stochastic Training of Neural Fields](https://mlsanity.com/paper/2402.08784/) (2024)
- [Preserving Full Degradation Details for Blind Image Super-Resolution](https://mlsanity.com/paper/2407.01299/) (2024)

## Abstract (arXiv)

Blind face restoration (BFR) is fundamentally challenged by the extensive range of degradation types and degrees that impact model generalization. Recent advancements in diffusion models have made considerable progress in this field. Nevertheless, a critical limitation is their lack of awareness of specific degradation, leading to potential issues such as unnatural details and inaccurate textures. In this paper, we equip diffusion models with the capability to decouple various degradation as a degradation prompt from low-quality (LQ) face images via unsupervised contrastive learning with reconstruction loss, and demonstrate that this capability significantly improves performance, particularly in terms of the naturalness of the restored images. Our novel restoration scheme, named DR-BFR, guides the denoising of Latent Diffusion Models (LDM) by incorporating Degradation Representation (DR) and content features from LQ images. DR-BFR comprises two modules: 1) Degradation Representation Module (DRM): This module extracts degradation representation with content-irrelevant features from LQ faces and estimates a reasonable distribution in the degradation space through contrastive learning and a specially designed LQ reconstruction. 2) Latent Diffusion Restoration Module (LDRM): This module perceives both degradation features and content features in the latent space, enabling the restoration of high-quality images from LQ inputs. Our experiments demonstrate that the proposed DR-BFR significantly outperforms state-of-the-art methods quantitatively and qualitatively across various datasets. The DR effectively distinguishes between various degradations in blind face inverse problems and provides a reasonably powerful prompt to LDM.