# Overcoming False Illusions in Real-World Face Restoration with Multi-Modal Guided Diffusion Model arXiv: https://arxiv.org/abs/2410.04161 ยท HTML: https://mlsanity.com/paper/2410.04161/ ## Key facts - Cited by 12 papers in the map (data through 14 Oct 2026). - Placed in Inverse problems and image restoration, one of 14 directions in Diffusion models. - Builds on 43 of its 55 references that are in the map. - First posted on arXiv 5 Oct 2024. ## Placed in - Diffusion models / [Inverse problems and image restoration](https://mlsanity.com/diffusion/inverse-problems-and-image-restoration/) ## Builds on (most cited first, 30 of 43) - [Learning Transferable Visual Models From Natural Language Supervision](https://mlsanity.com/paper/2103.00020/) (2021) - [U-Net: Convolutional Networks for Biomedical Image Segmentation](https://mlsanity.com/paper/1505.04597/) (2015) - [High-Resolution Image Synthesis with Latent Diffusion Models](https://mlsanity.com/paper/2112.10752/) (2021) - [The Unreasonable Effectiveness of Deep Features as a Perceptual Metric](https://mlsanity.com/paper/1801.03924/) (2018) - [A Style-Based Generator Architecture for Generative Adversarial Networks](https://mlsanity.com/paper/1812.04948/) (2018) - [Classifier-Free Diffusion Guidance](https://mlsanity.com/paper/2207.12598/) (2022) - [Adding Conditional Control to Text-to-Image Diffusion Models](https://mlsanity.com/paper/2302.05543/) (2023) - [Deep Learning Face Attributes in the Wild](https://mlsanity.com/paper/1411.7766/) (2014) - [Analyzing and Improving the Image Quality of StyleGAN](https://mlsanity.com/paper/1912.04958/) (2019) - [Enhanced Deep Residual Networks for Single Image Super-Resolution](https://mlsanity.com/paper/1707.02921/) (2017) - [Beyond a Gaussian Denoiser: Residual Learning of Deep CNN for Image Denoising](https://mlsanity.com/paper/1608.03981/) (2016) - [T2I-Adapter: Learning Adapters to Dig out More Controllable Ability for Text-to-Image Diffusion Models](https://mlsanity.com/paper/2302.08453/) (2023) - [MUSIQ: Multi-scale Image Quality Transformer](https://mlsanity.com/paper/2108.05997/) (2021) - [Exploring CLIP for Assessing the Look and Feel of Images](https://mlsanity.com/paper/2207.12396/) (2022) - [VGGFace2: A dataset for recognising faces across pose and age](https://mlsanity.com/paper/1710.08092/) (2017) - [Interpreting the Latent Space of GANs for Semantic Face Editing](https://mlsanity.com/paper/1907.10786/) (2019) - [MANIQA: Multi-dimension Attention Network for No-Reference Image Quality Assessment](https://mlsanity.com/paper/2204.08958/) (2022) - [Learning Face Representation from Scratch](https://mlsanity.com/paper/1411.7923/) (2014) - [DeblurGAN: Blind Motion Deblurring Using Conditional Adversarial Networks](https://mlsanity.com/paper/1711.07064/) (2017) - [PULSE: Self-Supervised Photo Upsampling via Latent Space Exploration of Generative Models](https://mlsanity.com/paper/2003.03808/) (2020) - [Towards Real-World Blind Face Restoration with Generative Facial Prior](https://mlsanity.com/paper/2101.04061/) (2021) - [Scaling Up to Excellence: Practicing Model Scaling for Photo-Realistic Image Restoration In the Wild](https://mlsanity.com/paper/2401.13627/) (2024) - [DiffBIR: Towards Blind Image Restoration with Generative Diffusion Prior](https://mlsanity.com/paper/2308.15070/) (2023) - [Towards Robust Blind Face Restoration with Codebook Lookup Transformer](https://mlsanity.com/paper/2206.11253/) (2022) - [GAN Prior Embedded Network for Blind Face Restoration in the Wild](https://mlsanity.com/paper/2105.06070/) (2021) - [Image Processing Using Multi-Code GAN Prior](https://mlsanity.com/paper/1912.07116/) (2019) - [PIPAL: a Large-Scale Image Quality Assessment Dataset for Perceptual Image Restoration](https://mlsanity.com/paper/2007.12142/) (2020) - [FSRNet: End-to-End Learning Face Super-Resolution with Facial Priors](https://mlsanity.com/paper/1711.10703/) (2017) - [GLEAN: Generative Latent Bank for Large-Factor Image Super-Resolution](https://mlsanity.com/paper/2012.00739/) (2020) - [Compression Artifacts Reduction by a Deep Convolutional Network](https://mlsanity.com/paper/1504.06993/) (2015) ## Built on by (most cited first, 12 of 12) - [The Second Challenge on Real-World Face Restoration at NTIRE 2026: Methods and Results](https://mlsanity.com/paper/2604.10532/) (2026) - [Show and Polish: Reference-Guided Identity Preservation in Face Video Restoration](https://mlsanity.com/paper/2507.10293/) (2025) - [NTIRE 2025 Challenge on Real-World Face Restoration: Methods and Results](https://mlsanity.com/paper/2504.14600/) (2025) - [UniCon: Unidirectional Information Flow for Effective Control of Large-Scale Diffusion Models](https://mlsanity.com/paper/2503.17221/) (2025) - [Robust ID-Specific Face Restoration via Alignment Learning](https://mlsanity.com/paper/2507.10943/) (2025) - [Personalized Generative Low-light Image Denoising and Enhancement](https://mlsanity.com/paper/2412.14327/) (2024) - [HonestFace: Towards Honest Face Restoration with One-Step Diffusion Model](https://mlsanity.com/paper/2505.18469/) (2025) - [RefSTAR: Blind Facial Image Restoration with Reference Selection, Transfer, and Reconstruction](https://mlsanity.com/paper/2507.10470/) (2025) - [Personalized Face Super-Resolution with Identity Decoupling and Fitting](https://mlsanity.com/paper/2508.10937/) (2025) - [MANI-Pure: Magnitude-Adaptive Noise Injection for Adversarial Purification](https://mlsanity.com/paper/2509.25082/) (2025) - [Face2Scene: Using Facial Degradation as an Oracle for Diffusion-Based Scene Restoration](https://mlsanity.com/paper/2603.16570/) (2026) - [A2BFR: Attribute-Aware Blind Face Restoration](https://mlsanity.com/paper/2603.29423/) (2026) ## Abstract (arXiv) We introduce a novel Multi-modal Guided Real-World Face Restoration (MGFR) technique designed to improve the quality of facial image restoration from low-quality inputs. Leveraging a blend of attribute text prompts, high-quality reference images, and identity information, MGFR can mitigate the generation of false facial attributes and identities often associated with generative face restoration methods. By incorporating a dual-control adapter and a two-stage training strategy, our method effectively utilizes multi-modal prior information for targeted restoration tasks. We also present the Reface-HQ dataset, comprising over 21,000 high-resolution facial images across 4800 identities, to address the need for reference face training images. Our approach achieves superior visual quality in restoring facial details under severe degradation and allows for controlled restoration processes, enhancing the accuracy of identity preservation and attribute correction. Including negative quality samples and attribute prompts in the training further refines the model's ability to generate detailed and perceptually accurate images.