# FaithDiff: Unleashing Diffusion Priors for Faithful Image Super-resolution

arXiv: https://arxiv.org/abs/2411.18824 · HTML: https://mlsanity.com/paper/2411.18824/

## Key facts

- Cited by 32 papers in the map (data through 14 Oct 2026).
- Placed in Inverse problems and image restoration, one of 14 directions in Diffusion models.
- Builds on 46 of its 54 references that are in the map.
- First posted on arXiv 27 Nov 2024.

## Placed in

- Diffusion models / [Inverse problems and image restoration](https://mlsanity.com/diffusion/inverse-problems-and-image-restoration/)

## Builds on (most cited first, 30 of 46)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [Learning Transferable Visual Models From Natural Language Supervision](https://mlsanity.com/paper/2103.00020/) (2021)
- [Denoising Diffusion Probabilistic Models](https://mlsanity.com/paper/2006.11239/) (2020)
- [High-Resolution Image Synthesis with Latent Diffusion Models](https://mlsanity.com/paper/2112.10752/) (2021)
- [The Unreasonable Effectiveness of Deep Features as a Perceptual Metric](https://mlsanity.com/paper/1801.03924/) (2018)
- [Visual Instruction Tuning](https://mlsanity.com/paper/2304.08485/) (2023)
- [Hierarchical Text-Conditional Image Generation with CLIP Latents](https://mlsanity.com/paper/2204.06125/) (2022)
- [Photorealistic Text-to-Image Diffusion Models with Deep Language Understanding](https://mlsanity.com/paper/2205.11487/) (2022)
- [A Style-Based Generator Architecture for Generative Adversarial Networks](https://mlsanity.com/paper/1812.04948/) (2018)
- [Classifier-Free Diffusion Guidance](https://mlsanity.com/paper/2207.12598/) (2022)
- [Adding Conditional Control to Text-to-Image Diffusion Models](https://mlsanity.com/paper/2302.05543/) (2023)
- [SGDR: Stochastic Gradient Descent with Warm Restarts](https://mlsanity.com/paper/1608.03983/) (2016)
- [SDXL: Improving Latent Diffusion Models for High-Resolution Image Synthesis](https://mlsanity.com/paper/2307.01952/) (2023)
- [Elucidating the Design Space of Diffusion-Based Generative Models](https://mlsanity.com/paper/2206.00364/) (2022)
- [Enhanced Deep Residual Networks for Single Image Super-Resolution](https://mlsanity.com/paper/1707.02921/) (2017)
- [T2I-Adapter: Learning Adapters to Dig out More Controllable Ability for Text-to-Image Diffusion Models](https://mlsanity.com/paper/2302.08453/) (2023)
- [MUSIQ: Multi-scale Image Quality Transformer](https://mlsanity.com/paper/2108.05997/) (2021)
- [Exploring CLIP for Assessing the Look and Feel of Images](https://mlsanity.com/paper/2207.12396/) (2022)
- [Real-ESRGAN: Training Real-World Blind Super-Resolution with Pure Synthetic Data](https://mlsanity.com/paper/2107.10833/) (2021)
- [PixArt-α: Fast Training of Diffusion Transformer for Photorealistic Text-to-Image Synthesis](https://mlsanity.com/paper/2310.00426/) (2023)
- [Animate Anyone: Consistent and Controllable Image-to-Video Synthesis for Character Animation](https://mlsanity.com/paper/2311.17117/) (2023)
- [Designing a Practical Degradation Model for Deep Blind Image Super-Resolution](https://mlsanity.com/paper/2103.14006/) (2021)
- [MANIQA: Multi-dimension Attention Network for No-Reference Image Quality Assessment](https://mlsanity.com/paper/2204.08958/) (2022)
- [Exploiting Diffusion Prior for Real-World Image Super-Resolution](https://mlsanity.com/paper/2305.07015/) (2023)
- [ResShift: Efficient Diffusion Model for Image Super-resolution by Residual Shifting](https://mlsanity.com/paper/2307.12348/) (2023)
- [Towards Real-World Blind Face Restoration with Generative Facial Prior](https://mlsanity.com/paper/2101.04061/) (2021)
- [SeeSR: Towards Semantics-Aware Real-World Image Super-Resolution](https://mlsanity.com/paper/2311.16518/) (2023)
- [Scaling Up to Excellence: Practicing Model Scaling for Photo-Realistic Image Restoration In the Wild](https://mlsanity.com/paper/2401.13627/) (2024)
- [DiffBIR: Towards Blind Image Restoration with Generative Diffusion Prior](https://mlsanity.com/paper/2308.15070/) (2023)
- [Pixel-Aware Stable Diffusion for Realistic Image Super-resolution and Personalized Stylization](https://mlsanity.com/paper/2308.14469/) (2023)

## Built on by (most cited first, 30 of 32)

- [Enhancing Text-to-Image Diffusion Transformer via Split-Text Conditioning](https://mlsanity.com/paper/2505.19261/) (2025)
- [Text-Aware Image Restoration with Diffusion Models](https://mlsanity.com/paper/2506.09993/) (2025)
- [GenDR: Lighten Generative Detail Restoration](https://mlsanity.com/paper/2503.06790/) (2025)
- [SRHand: Super-Resolving Hand Images and 3D Shapes via View/Pose-aware Neural Image Representations and Explicit 3D Meshes](https://mlsanity.com/paper/2509.21859/) (2025)
- [Joint Geometric and Trajectory Consistency Learning for One-Step Real-World Super-Resolution](https://mlsanity.com/paper/2602.24240/) (2026)
- [Next-Frame Decoding for Ultra-Low-Bitrate Image Compression with Video Diffusion Priors](https://mlsanity.com/paper/2603.15129/) (2026)
- [Enhanced Semantic Extraction and Guidance for UGC Image Super Resolution](https://mlsanity.com/paper/2504.09887/) (2025)
- [Eliminating VAE for Fast and High-Resolution Generative Detail Restoration](https://mlsanity.com/paper/2602.10630/) (2026)
- [FluxFlow: Conservative Flow-Matching for Astronomical Image Super-Resolution](https://mlsanity.com/paper/2605.03749/) (2026)
- [OP4KSR: One-Step Patch-Free 4K Super-Resolution with Periodic Artifact Suppression](https://mlsanity.com/paper/2605.13457/) (2026)
- [MGCR-Net:Multimodal Graph-Conditioned Vision-Language Reconstruction Network for Remote Sensing Change Detection](https://mlsanity.com/paper/2508.01555/) (2025)
- [OmniScaleSR: Unleashing Scale-Controlled Diffusion Prior for Faithful and Realistic Arbitrary-Scale Image Super-Resolution](https://mlsanity.com/paper/2512.04699/) (2025)
- [Bird-SR: Bidirectional Reward-Guided Diffusion for Real-World Image Super-Resolution](https://mlsanity.com/paper/2602.07069/) (2026)
- [Unified Removal of Raindrops and Reflections: A New Benchmark and A Novel Pipeline](https://mlsanity.com/paper/2603.16446/) (2026)
- [Tuning Real-World Image Restoration at Inference: A Test-Time Scaling Paradigm for Flow Matching Models](https://mlsanity.com/paper/2603.22027/) (2026)
- [Coloring the Noise: Adversarial Sobolev Alignment for Faithful Image Super Resolution](https://mlsanity.com/paper/2605.23264/) (2026)
- [GGT-100K: Generative Ground Truth for Generalizable Real-World Image Restoration](https://mlsanity.com/paper/2605.31039/) (2026)
- [MFSR: Multi-fractal Feature for Super-resolution Reconstruction with Fine Details Recovery](https://mlsanity.com/paper/2502.19797/) (2025)
- [MegaSR: Mining Customized Semantics and Expressive Guidance for Real-World Image Super-Resolution](https://mlsanity.com/paper/2503.08096/) (2025)
- [LAFR: Efficient Diffusion-based Blind Face Restoration via Latent Codebook Alignment Adapter](https://mlsanity.com/paper/2505.23462/) (2025)
- [AngularFuse: A Closer Look at Angle-based Perception for Spatial-Sensitive Multi-Modality Image Fusion](https://mlsanity.com/paper/2510.12260/) (2025)
- [Boosting Fidelity for Pre-Trained-Diffusion-Based Low-Light Image Enhancement via Condition Refinement](https://mlsanity.com/paper/2510.17105/) (2025)
- [EvDiff: Event-Based Video Reconstruction using One-Step Diffusion Models](https://mlsanity.com/paper/2511.17492/) (2025)
- [Joint Degradation-Aware Arbitrary-Scale Super-Resolution for Variable-Rate Extreme Image Compression](https://mlsanity.com/paper/2603.17408/) (2026)
- [Learning to Balance: Decoupled Siamese Diffusion Transformer for Reference-Based Remote Sensing Image Super-Resolution](https://mlsanity.com/paper/2605.17980/) (2026)
- [VICR: Visual In-Context Restoration for Real-World Image Super-Resolution](https://mlsanity.com/paper/2606.00704/) (2026)
- [When Latents Forget Pixels: Restoring Fidelity in Diffusion Transformer Super-Resolution](https://mlsanity.com/paper/2608.09133/) (2026)
- [MagnifiQ: Patch-aware Text Guided Progressive Upscaling for High-Resolution Image Restoration](https://mlsanity.com/paper/2608.14543/) (2026)
- [Uncertainty-Guided Latent Diffusion Models for Faithful Super Resolution](https://mlsanity.com/paper/2608.25998/) (2026)
- [In-Token Learning for High-Fidelity Image Restoration via Diffusion Transformers](https://mlsanity.com/paper/2609.33523/) (2026)

## Abstract (arXiv)

Faithful image super-resolution (SR) not only needs to recover images that appear realistic, similar to image generation tasks, but also requires that the restored images maintain fidelity and structural consistency with the input. To this end, we propose a simple and effective method, named FaithDiff, to fully harness the impressive power of latent diffusion models (LDMs) for faithful image SR. In contrast to existing diffusion-based SR methods that freeze the diffusion model pre-trained on high-quality images, we propose to unleash the diffusion prior to identify useful information and recover faithful structures. As there exists a significant gap between the features of degraded inputs and the noisy latent from the diffusion model, we then develop an effective alignment module to explore useful features from degraded inputs to align well with the diffusion process. Considering the indispensable roles and interplay of the encoder and diffusion model in LDMs, we jointly fine-tune them in a unified optimization framework, facilitating the encoder to extract useful features that coincide with diffusion process. Extensive experimental results demonstrate that FaithDiff outperforms state-of-the-art methods, providing high-quality and faithful SR results.