Your Pre-trained Diffusion Model Secretly Knows Restoration
On the map: Diffusion models → Inverse problems and image restoration
Abstract (arXiv)
Pre-trained diffusion models have enabled significant advancements in All-in-One Restoration (AiOR), offering improved perceptual quality and generalization. However, diffusion-based restoration methods primarily rely on fine-tuning or Control-Net style modules to leverage the pre-trained diffusion model's priors for AiOR. In this work, we show that these pre-trained diffusion models inherently possess restoration behavior, which can be unlocked by directly learning prompt embeddings at the output of the text encoder. Interestingly, this behavior is largely inaccessible through text prompts and text-token embedding optimization. Furthermore, we observe that naive prompt learning is unstable because the forward noising process using degraded images is misaligned with the reverse sampling trajectory. To resolve this, we train prompts within a diffusion bridge formulation that aligns training and inference dynamics, enforcing a coherent denoising path from noisy degraded states to clean images. Building on these insights, we introduce our lightweight learned prompts on the pre-trained WAN video model and FLUX image models, converting them into high-performing restoration models. Extensive experiments demonstrate that our approach achieves competitive performance and generalization across diverse degradations, while avoiding fine-tuning and restoration-specific control modules.
Key facts. No citing papers in the map yet (data through 14 Oct 2026). Placed in Inverse problems and image restoration, one of 14 directions in Diffusion models. Builds on 48 of its 63 references that are in the map. First posted on arXiv 6 Apr 2026.
Builds on
48 of 63 references.
Closest ancestors · 7
Same direction, newest first
- 2025UniLDiff: Unlocking the Power of Diffusion Priors for All-in-One Image Restoration31 Jul
- 2025UniCoRN: Latent Diffusion-based Unified Controllable Image Restoration Network across Multiple Degradations20 Mar
- 2024GenDeg: Diffusion-based Degradation Synthesis for Generalizable All-In-One Image Restoration26 Nov
- 2024PnP-Flow: Plug-and-Play Image Restoration with Flow Matching3 Oct
- 2024Diff-Restorer: Unleashing Visual Prompts for Diffusion-based Universal Image Restoration4 Jul
- 2023AutoDIR: Automatic All-in-One Image Restoration with Latent Diffusion16 Oct
- 2023Generative Diffusion Prior for Unified Image Restoration and Enhancement3 Apr
Other references · 41
- 2021High-Resolution Image Synthesis with Latent Diffusion Modelscited by 18,863
- 2018The Unreasonable Effectiveness of Deep Features as a Perceptual Metriccited by 11,177
- 2020Denoising Diffusion Implicit Modelscited by 9,781
- 2023SDXL: Improving Latent Diffusion Models for High-Resolution Image Synthesiscited by 4,356
- 2025Wan: Open and Advanced Large-Scale Video Generative Modelscited by 2,708
- 2022Prompt-to-Prompt Image Editing with Cross Attention Controlcited by 2,431
- 2021SDEdit: Guided Image Synthesis and Editing with Stochastic Differential Equationscited by 1,937
- 2022Null-text Inversion for Editing Real Images using Guided Diffusion Modelscited by 1,179
- 2021MUSIQ: Multi-scale Image Quality Transformercited by 1,081
- 2022Exploring CLIP for Assessing the Look and Feel of Imagescited by 991
- 2022Plug-and-Play Diffusion Features for Text-Driven Image-to-Image Translationcited by 906
- 2021Multi-Stage Progressive Image Restorationcited by 816
29 more
- 2020Image Quality Assessment: Unifying Structure and Texture Similaritycited by 811
- 2016Deep Multi-scale Convolutional Neural Network for Dynamic Scene Deblurringcited by 727
- 2023MasaCtrl: Tuning-Free Mutual Self-Attention Control for Consistent Image Synthesis and Editingcited by 727
- 2018Deep Retinex Decomposition for Low-Light Enhancementcited by 723
- 2017Benchmarking Single Image Dehazing and Beyondcited by 447
- 2022Exploring Video Quality Assessment on User Generated Contents from Aesthetic and Technical Perspectivescited by 320
- 2021TransWeather: Transformer-based Restoration of Images Degraded by Adverse Weather Conditionscited by 244
- 2017DesnowNet: Context-Aware Deep Network for Snow Removalcited by 200
- 2023Denoising Diffusion Bridge Modelscited by 178
- 2021Low-Light Image and Video Enhancement Using Deep Learning: A Surveycited by 148
- 2024InstructIR: High-Quality Image Restoration Following Human Instructionscited by 140
- 2024Lumina-T2X: Transforming Text into Any Modality, Resolution, and Duration via Flow-based Large Diffusion Transformerscited by 137
- 2024Lumina-Next: Making Lumina-T2X Stronger and Faster with Next-DiTcited by 132
- 2022LEDNet: Joint Low-light Enhancement and Deblurring in the Darkcited by 96
- 2024OneRestore: A Universal Restoration Framework for Composite Degradationcited by 92
- 2018Robust Video Content Alignment and Compensation for Rain Removal in a CNN Frameworkcited by 53
- 2024Diffusion Bridge Implicit Modelscited by 49
- 2024Diff-Plugin: Revitalizing Details for Diffusion-based Low-level Taskscited by 48
- 2020Image Restoration for Under-Display Cameracited by 47
- 2025Degradation-Aware Feature Perturbation for All-in-One Image Restorationcited by 41
- 2023From Sky to the Ground: A Large-scale Benchmark and Simple Baseline Towards Real Rain Removalcited by 39
- 2024FoundIR: Unleashing Million-scale Training Data to Advance Foundation Models for Image Restorationcited by 39
- 2025Universal Image Restoration Pre-training via Degradation Classificationcited by 35
- 2024PixWizard: Versatile Image-to-Image Visual Assistant with Open-Language Instructionscited by 31
- 2023Video Adverse-Weather-Component Suppression Network via Weather Messenger and Adversarial Backpropagationcited by 21
- 2025WeatherBench: A Real-World Benchmark Dataset for All-in-One Adverse Weather Image Restorationcited by 20
- 2024AWRaCLe: All-Weather Image Restoration using Visual In-Context Learningcited by 18
- 2025RestoreVAR: Visual Autoregressive Generation for All-in-One Image Restorationcited by 15
- 2019Removing Rain in Videos: A Large-scale Database and A Two-stream ConvLSTM Approachcited by 4
Built on by
Not cited yet.
Similar, not cited
Close in wording, no citation either way
- 2025Elucidating and Endowing the Diffusion Training Paradigm for General Image Restoration
- 2023Multimodal Prompt Perceiver: Empower Adaptiveness, Generalizability and Fidelity for All-in-One Image Restoration
- 2026In-Token Learning for High-Fidelity Image Restoration via Diffusion Transformers
- 2025Harnessing Diffusion-Yielded Score Priors for Image Restoration
- 2025Dual Prompting Image Restoration with Diffusion Transformers
- 2026PAST: Prompt-Adaptive Sampling Termination for Efficient Diffusion Model
- 2023Prompt-tuning latent diffusion models for inverse problems
- 2025TDM: Temporally-Consistent Diffusion Model for All-in-One Real-World Video Restoration