# Harnessing Diffusion-Yielded Score Priors for Image Restoration

arXiv: https://arxiv.org/abs/2507.20590 · HTML: https://mlsanity.com/paper/2507.20590/

## Key facts

- Cited by 36 papers in the map (data through 14 Oct 2026).
- Placed in Inverse problems and image restoration, one of 14 directions in Diffusion models.
- Builds on 58 of its 100 references that are in the map.
- First posted on arXiv 28 Jul 2025.

## Placed in

- Diffusion models / [Inverse problems and image restoration](https://mlsanity.com/diffusion/inverse-problems-and-image-restoration/)

## Builds on (most cited first, 30 of 58)

- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [Learning Transferable Visual Models From Natural Language Supervision](https://mlsanity.com/paper/2103.00020/) (2021)
- [Denoising Diffusion Probabilistic Models](https://mlsanity.com/paper/2006.11239/) (2020)
- [Denoising Diffusion Implicit Models](https://mlsanity.com/paper/2010.02502/) (2020)
- [Diffusion Models Beat GANs on Image Synthesis](https://mlsanity.com/paper/2105.05233/) (2021)
- [Emerging Properties in Self-Supervised Vision Transformers](https://mlsanity.com/paper/2104.14294/) (2021)
- [Hierarchical Text-Conditional Image Generation with CLIP Latents](https://mlsanity.com/paper/2204.06125/) (2022)
- [A Style-Based Generator Architecture for Generative Adversarial Networks](https://mlsanity.com/paper/1812.04948/) (2018)
- [SDXL: Improving Latent Diffusion Models for High-Resolution Image Synthesis](https://mlsanity.com/paper/2307.01952/) (2023)
- [Zero-Shot Text-to-Image Generation](https://mlsanity.com/paper/2102.12092/) (2021)
- [Scaling Rectified Flow Transformers for High-Resolution Image Synthesis](https://mlsanity.com/paper/2403.03206/) (2024)
- [Generative Modeling by Estimating Gradients of the Data Distribution](https://mlsanity.com/paper/1907.05600/) (2019)
- [Progressive Growing of GANs for Improved Quality, Stability, and Variation](https://mlsanity.com/paper/1710.10196/) (2017)
- [Improved Techniques for Training GANs](https://mlsanity.com/paper/1606.03498/) (2016)
- [Large Scale GAN Training for High Fidelity Natural Image Synthesis](https://mlsanity.com/paper/1809.11096/) (2018)
- [Improved Training of Wasserstein GANs](https://mlsanity.com/paper/1704.00028/) (2017)
- [Unsupervised Representation Learning with Deep Convolutional Generative Adversarial Networks](https://mlsanity.com/paper/1511.06434/) (2015)
- [Photo-Realistic Single Image Super-Resolution Using a Generative Adversarial Network](https://mlsanity.com/paper/1609.04802/) (2016)
- [SwinIR: Image Restoration Using Swin Transformer](https://mlsanity.com/paper/2108.10257/) (2021)
- [Image Super-Resolution via Iterative Refinement](https://mlsanity.com/paper/2104.07636/) (2021)
- [Diffusion Posterior Sampling for General Noisy Inverse Problems](https://mlsanity.com/paper/2209.14687/) (2022)
- [ESRGAN: Enhanced Super-Resolution Generative Adversarial Networks](https://mlsanity.com/paper/1809.00219/) (2018)
- [Image Super-Resolution Using Deep Convolutional Networks](https://mlsanity.com/paper/1501.00092/) (2014)
- [MUSIQ: Multi-scale Image Quality Transformer](https://mlsanity.com/paper/2108.05997/) (2021)
- [One-step Diffusion with Distribution Matching Distillation](https://mlsanity.com/paper/2311.18828/) (2023)
- [Exploring CLIP for Assessing the Look and Feel of Images](https://mlsanity.com/paper/2207.12396/) (2022)
- [Real-ESRGAN: Training Real-World Blind Super-Resolution with Pure Synthetic Data](https://mlsanity.com/paper/2107.10833/) (2021)
- [Denoising Diffusion Restoration Models](https://mlsanity.com/paper/2201.11793/) (2022)
- [Which Training Methods for GANs do actually Converge?](https://mlsanity.com/paper/1801.04406/) (2018)
- [Conditional Image Synthesis With Auxiliary Classifier GANs](https://mlsanity.com/paper/1610.09585/) (2016)

## Built on by (most cited first, 30 of 36)

- [PhotoFramer: Multi-modal Image Composition Instruction](https://mlsanity.com/paper/2512.00993/) (2025)
- [FoundIR-v2: Optimizing Pre-Training Data Mixtures for Image Restoration Foundation Model](https://mlsanity.com/paper/2512.09282/) (2025)
- [Improved Adversarial Diffusion Compression for Real-World Video Super-Resolution](https://mlsanity.com/paper/2603.00458/) (2026)
- [Can Nano Banana 2 Replace Traditional Image Restoration Models? An Evaluation of Its Performance on Image Restoration Tasks](https://mlsanity.com/paper/2604.03061/) (2026)
- [Bridging Fidelity-Reality with Controllable One-Step Diffusion for Image Super-Resolution](https://mlsanity.com/paper/2512.14061/) (2025)
- [Joint Geometric and Trajectory Consistency Learning for One-Step Real-World Super-Resolution](https://mlsanity.com/paper/2602.24240/) (2026)
- [Condition Matters in Full-head 3D GANs](https://mlsanity.com/paper/2602.07198/) (2026)
- [How far have we gone in Generative Image Restoration? A study on its capability, limitations and evaluation practices](https://mlsanity.com/paper/2603.05010/) (2026)
- [Spectral and Trajectory Regularization for Diffusion Transformer Super-Resolution](https://mlsanity.com/paper/2603.06275/) (2026)
- [MFSR: MeanFlow Distillation for One Step Real-World Image Super Resolution](https://mlsanity.com/paper/2603.20690/) (2026)
- [OP4KSR: One-Step Patch-Free 4K Super-Resolution with Periodic Artifact Suppression](https://mlsanity.com/paper/2605.13457/) (2026)
- [Position: Evaluation of Visual Processing Should Be Human-Centered, Not Metric-Centered](https://mlsanity.com/paper/2603.00643/) (2026)
- [OARS: Process-Aware Online Alignment for Generative Real-World Image Super-Resolution](https://mlsanity.com/paper/2603.12811/) (2026)
- [FUMO: Prior-Modulated Diffusion for Single Image Reflection Removal](https://mlsanity.com/paper/2603.19036/) (2026)
- [Bridging Restoration and Generation in One-step Diffusion for Real-World Image Super-Resolution](https://mlsanity.com/paper/2604.24136/) (2026)
- [Adding Thermal Awareness to Visual Systems in Real-Time via Distilled Diffusion Models](https://mlsanity.com/paper/2605.06010/) (2026)
- [Noise-Started One-Step Real-World Super-Resolution via LR-Conditioned SplitMeanFlow and GAN Refinement](https://mlsanity.com/paper/2605.09328/) (2026)
- [PRISM: Prior Rectification and Uncertainty-Aware Structure Modeling for Diffusion-Based Text Image Super-Resolution](https://mlsanity.com/paper/2605.13027/) (2026)
- [DreamSR: Towards Ultra-High-Resolution Image Super-Resolution via a Receptive-Field Enhanced Diffusion Transformer](https://mlsanity.com/paper/2605.15682/) (2026)
- [TEASR: Training-Efficient Any-Step Diffusion Transformer for Real-World Image Super-Resolution](https://mlsanity.com/paper/2606.16188/) (2026)
- [DNF-SR: Dual-Input and Negative-Aware Feature Fine-Tuning for Real-World Image Super-Resolution](https://mlsanity.com/paper/2609.15120/) (2026)
- [The Unanticipated Asymmetry Between Perceptual Optimization and Assessment](https://mlsanity.com/paper/2509.20878/) (2025)
- [LucidFlux: Caption-Free Photo-Realistic Image Restoration via a Large-Scale Diffusion Transformer](https://mlsanity.com/paper/2509.22414/) (2025)
- [PGP-DiffSR: Phase-Guided Progressive Pruning for Efficient Diffusion-based Image Super-Resolution](https://mlsanity.com/paper/2512.02681/) (2025)
- [Generative Multi-Focus Image Fusion](https://mlsanity.com/paper/2512.21495/) (2025)
- [gQIR: Generative Quanta Image Reconstruction](https://mlsanity.com/paper/2602.20417/) (2026)
- [RefReward-SR: LR-Conditioned Reward Modeling for Preference-Aligned Super-Resolution](https://mlsanity.com/paper/2603.24198/) (2026)
- [DF3DV-1K: A Large-Scale Dataset and Benchmark for Distractor-Free Novel View Synthesis](https://mlsanity.com/paper/2604.13416/) (2026)
- [ReImagine: Rethinking Controllable High-Quality Human Video Generation via Image-First Synthesis](https://mlsanity.com/paper/2604.19720/) (2026)
- [OSOR: One-Step Diffusion Inpainting for Effect-Aware Object Removal](https://mlsanity.com/paper/2606.28094/) (2026)

## Abstract (arXiv)

Deep image restoration models aim to learn a mapping from degraded image space to natural image space. However, they face several critical challenges: removing degradation, generating realistic details, and ensuring pixel-level consistency. Over time, three major classes of methods have emerged, including MSE-based, GAN-based, and diffusion-based methods. However, they fail to achieve a good balance between restoration quality, fidelity, and speed. We propose a novel method, HYPIR, to address these challenges. Our solution pipeline is straightforward: it involves initializing the image restoration model with a pre-trained diffusion model and then fine-tuning it with adversarial training. This approach does not rely on diffusion loss, iterative sampling, or additional adapters. We theoretically demonstrate that initializing adversarial training from a pre-trained diffusion model positions the initial restoration model very close to the natural image distribution. Consequently, this initialization improves numerical stability, avoids mode collapse, and substantially accelerates the convergence of adversarial training. Moreover, HYPIR inherits the capabilities of diffusion models with rich user control, enabling text-guided restoration and adjustable texture richness. Requiring only a single forward pass, it achieves faster convergence and inference speed than diffusion-based methods. Extensive experiments show that HYPIR outperforms previous state-of-the-art methods, achieving efficient and high-quality image restoration.