mlsanityData to 14 Oct 2026

Your Pre-trained Diffusion Model Secretly Knows Restoration

arXiv 2604.04924 ↗First posted 6 Apr 2026cs.CV cs.AIcited by 0

On the map: Diffusion models → Inverse problems and image restoration

Abstract (arXiv)

Pre-trained diffusion models have enabled significant advancements in All-in-One Restoration (AiOR), offering improved perceptual quality and generalization. However, diffusion-based restoration methods primarily rely on fine-tuning or Control-Net style modules to leverage the pre-trained diffusion model's priors for AiOR. In this work, we show that these pre-trained diffusion models inherently possess restoration behavior, which can be unlocked by directly learning prompt embeddings at the output of the text encoder. Interestingly, this behavior is largely inaccessible through text prompts and text-token embedding optimization. Furthermore, we observe that naive prompt learning is unstable because the forward noising process using degraded images is misaligned with the reverse sampling trajectory. To resolve this, we train prompts within a diffusion bridge formulation that aligns training and inference dynamics, enforcing a coherent denoising path from noisy degraded states to clean images. Building on these insights, we introduce our lightweight learned prompts on the pre-trained WAN video model and FLUX image models, converting them into high-performing restoration models. Extensive experiments demonstrate that our approach achieves competitive performance and generalization across diverse degradations, while avoiding fine-tuning and restoration-specific control modules.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Placed in Inverse problems and image restoration, one of 14 directions in Diffusion models. Builds on 48 of its 63 references that are in the map. First posted on arXiv 6 Apr 2026.

Builds on

48 of 63 references.

Closest ancestors · 7

Same direction, newest first

  1. 2025UniLDiff: Unlocking the Power of Diffusion Priors for All-in-One Image Restoration31 Jul
  2. 2025UniCoRN: Latent Diffusion-based Unified Controllable Image Restoration Network across Multiple Degradations20 Mar
  3. 2024GenDeg: Diffusion-based Degradation Synthesis for Generalizable All-In-One Image Restoration26 Nov
  4. 2024PnP-Flow: Plug-and-Play Image Restoration with Flow Matching3 Oct
  5. 2024Diff-Restorer: Unleashing Visual Prompts for Diffusion-based Universal Image Restoration4 Jul
  6. 2023AutoDIR: Automatic All-in-One Image Restoration with Latent Diffusion16 Oct
  7. 2023Generative Diffusion Prior for Unified Image Restoration and Enhancement3 Apr

Other references · 41

  1. 2021High-Resolution Image Synthesis with Latent Diffusion Modelscited by 18,863
  2. 2018The Unreasonable Effectiveness of Deep Features as a Perceptual Metriccited by 11,177
  3. 2020Denoising Diffusion Implicit Modelscited by 9,781
  4. 2023SDXL: Improving Latent Diffusion Models for High-Resolution Image Synthesiscited by 4,356
  5. 2025Wan: Open and Advanced Large-Scale Video Generative Modelscited by 2,708
  6. 2022Prompt-to-Prompt Image Editing with Cross Attention Controlcited by 2,431
  7. 2021SDEdit: Guided Image Synthesis and Editing with Stochastic Differential Equationscited by 1,937
  8. 2022Null-text Inversion for Editing Real Images using Guided Diffusion Modelscited by 1,179
  9. 2021MUSIQ: Multi-scale Image Quality Transformercited by 1,081
  10. 2022Exploring CLIP for Assessing the Look and Feel of Imagescited by 991
  11. 2022Plug-and-Play Diffusion Features for Text-Driven Image-to-Image Translationcited by 906
  12. 2021Multi-Stage Progressive Image Restorationcited by 816
29 more
  1. 2020Image Quality Assessment: Unifying Structure and Texture Similaritycited by 811
  2. 2016Deep Multi-scale Convolutional Neural Network for Dynamic Scene Deblurringcited by 727
  3. 2023MasaCtrl: Tuning-Free Mutual Self-Attention Control for Consistent Image Synthesis and Editingcited by 727
  4. 2018Deep Retinex Decomposition for Low-Light Enhancementcited by 723
  5. 2017Benchmarking Single Image Dehazing and Beyondcited by 447
  6. 2022Exploring Video Quality Assessment on User Generated Contents from Aesthetic and Technical Perspectivescited by 320
  7. 2021TransWeather: Transformer-based Restoration of Images Degraded by Adverse Weather Conditionscited by 244
  8. 2017DesnowNet: Context-Aware Deep Network for Snow Removalcited by 200
  9. 2023Denoising Diffusion Bridge Modelscited by 178
  10. 2021Low-Light Image and Video Enhancement Using Deep Learning: A Surveycited by 148
  11. 2024InstructIR: High-Quality Image Restoration Following Human Instructionscited by 140
  12. 2024Lumina-T2X: Transforming Text into Any Modality, Resolution, and Duration via Flow-based Large Diffusion Transformerscited by 137
  13. 2024Lumina-Next: Making Lumina-T2X Stronger and Faster with Next-DiTcited by 132
  14. 2022LEDNet: Joint Low-light Enhancement and Deblurring in the Darkcited by 96
  15. 2024OneRestore: A Universal Restoration Framework for Composite Degradationcited by 92
  16. 2018Robust Video Content Alignment and Compensation for Rain Removal in a CNN Frameworkcited by 53
  17. 2024Diffusion Bridge Implicit Modelscited by 49
  18. 2024Diff-Plugin: Revitalizing Details for Diffusion-based Low-level Taskscited by 48
  19. 2020Image Restoration for Under-Display Cameracited by 47
  20. 2025Degradation-Aware Feature Perturbation for All-in-One Image Restorationcited by 41
  21. 2023From Sky to the Ground: A Large-scale Benchmark and Simple Baseline Towards Real Rain Removalcited by 39
  22. 2024FoundIR: Unleashing Million-scale Training Data to Advance Foundation Models for Image Restorationcited by 39
  23. 2025Universal Image Restoration Pre-training via Degradation Classificationcited by 35
  24. 2024PixWizard: Versatile Image-to-Image Visual Assistant with Open-Language Instructionscited by 31
  25. 2023Video Adverse-Weather-Component Suppression Network via Weather Messenger and Adversarial Backpropagationcited by 21
  26. 2025WeatherBench: A Real-World Benchmark Dataset for All-in-One Adverse Weather Image Restorationcited by 20
  27. 2024AWRaCLe: All-Weather Image Restoration using Visual In-Context Learningcited by 18
  28. 2025RestoreVAR: Visual Autoregressive Generation for All-in-One Image Restorationcited by 15
  29. 2019Removing Rain in Videos: A Large-scale Database and A Two-stream ConvLSTM Approachcited by 4

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2025Elucidating and Endowing the Diffusion Training Paradigm for General Image Restoration
  2. 2023Multimodal Prompt Perceiver: Empower Adaptiveness, Generalizability and Fidelity for All-in-One Image Restoration
  3. 2026In-Token Learning for High-Fidelity Image Restoration via Diffusion Transformers
  4. 2025Harnessing Diffusion-Yielded Score Priors for Image Restoration
  5. 2025Dual Prompting Image Restoration with Diffusion Transformers
  6. 2026PAST: Prompt-Adaptive Sampling Termination for Efficient Diffusion Model
  7. 2023Prompt-tuning latent diffusion models for inverse problems
  8. 2025TDM: Temporally-Consistent Diffusion Model for All-in-One Real-World Video Restoration