mlsanityData to 7 Oct 2026

DiffIR: Efficient Diffusion Model for Image Restoration

arXiv 2303.09472 ↗First posted 16 Mar 2023cs.CVcited by 253

On the map: Diffusion models → Inverse problems and image restoration

Abstract (arXiv)

Diffusion model (DM) has achieved SOTA performance by modeling the image synthesis process into a sequential application of a denoising network. However, different from image synthesis, image restoration (IR) has a strong constraint to generate results in accordance with ground-truth. Thus, for IR, traditional DMs running massive iterations on a large model to estimate whole images or feature maps is inefficient. To address this issue, we propose an efficient DM for IR (DiffIR), which consists of a compact IR prior extraction network (CPEN), dynamic IR transformer (DIRformer), and denoising network. Specifically, DiffIR has two training stages: pretraining and training DM. In pretraining, we input ground-truth images into CPENS1 to capture a compact IR prior representation (IPR) to guide DIRformer. In the second stage, we train the DM to directly estimate the same IRP as pretrained CPENS1 only using LQ images. We observe that since the IPR is only a compact vector, DiffIR can use fewer iterations than traditional DM to obtain accurate estimations and generate more stable and realistic results. Since the iterations are few, our DiffIR can adopt a joint optimization of CPENS2, DIRformer, and denoising network, which can further reduce the estimation error influence. We conduct extensive experiments on several IR tasks and achieve SOTA performance while consuming less computational costs. Code is available at urlhttps://github.com/Zj-BinXia/DiffIR.

Key facts. Cited by 253 papers in the map (data through 7 Oct 2026). Placed in Inverse problems and image restoration, one of 14 directions in Diffusion models. Builds on 74 of its 89 references that are in the map. First posted on arXiv 16 Mar 2023.

Builds on

74 of 89 references.

Closest ancestors · 3

Same direction, newest first

  1. 2022Denoising Diffusion Restoration Models27 Jan
  2. 2021Come-Closer-Diffuse-Faster: Accelerating Conditional Diffusion Models for Inverse Problems through Stochastic Contraction9 Dec
  3. 2021SRDiff: Single Image Super-Resolution with Diffusion Probabilistic Models30 Apr

Other references · 71

  1. 2020An Image is Worth 16x16 Words: Transformers for Image Recognition at Scalecited by 29,915
  2. 2020Denoising Diffusion Probabilistic Modelscited by 20,851
  3. 2021High-Resolution Image Synthesis with Latent Diffusion Modelscited by 18,694
  4. 2018The Unreasonable Effectiveness of Deep Features as a Perceptual Metriccited by 11,029
  5. 2020Score-Based Generative Modeling through Stochastic Differential Equationscited by 8,964
  6. 2021Diffusion Models Beat GANs on Image Synthesiscited by 8,185
  7. 2020End-to-End Object Detection with Transformerscited by 7,384
  8. 2015Deep Unsupervised Learning using Nonequilibrium Thermodynamicscited by 7,005
  9. 2016Layer Normalizationcited by 5,579
  10. 2016SGDR: Stochastic Gradient Descent with Warm Restartscited by 4,993
  11. 2020Training data-efficient image transformers & distillation through attentioncited by 4,560
  12. 2020Deformable DETR: Deformable Transformers for End-to-End Object Detectioncited by 3,410
38 more
  1. 2017Progressive Growing of GANs for Improved Quality, Stability, and Variationcited by 2,967
  2. 2021SegFormer: Simple and Efficient Design for Semantic Segmentation with Transformerscited by 2,923
  3. 2017Improved Training of Wasserstein GANscited by 2,223
  4. 2014Conditional Generative Adversarial Netscited by 2,152
  5. 2021Pyramid Vision Transformer: A Versatile Backbone for Dense Prediction without Convolutionscited by 1,960
  6. 2016Photo-Realistic Single Image Super-Resolution Using a Generative Adversarial Networkcited by 1,898
  7. 2021SwinIR: Image Restoration Using Swin Transformercited by 1,869
  8. 2021Restormer: Efficient Transformer for High-Resolution Image Restorationcited by 1,618
  9. 2021Image Super-Resolution via Iterative Refinementcited by 1,506
  10. 2016Context Encoders: Feature Learning by Inpaintingcited by 1,424
  11. 2022RePaint: Inpainting using Denoising Diffusion Probabilistic Modelscited by 1,419
  12. 2020Rethinking Semantic Segmentation from a Sequence-to-Sequence Perspective with Transformerscited by 1,368
  13. 2016Beyond a Gaussian Denoiser: Residual Learning of Deep CNN for Image Denoisingcited by 1,352
  14. 2021Palette: Image-to-Image Diffusion Modelscited by 1,296
  15. 2018Image Super-Resolution Using Very Deep Residual Channel Attention Networkscited by 1,256
  16. 2018ESRGAN: Enhanced Super-Resolution Generative Adversarial Networkscited by 1,250
  17. 2014Image Super-Resolution Using Deep Convolutional Networkscited by 1,218
  18. 2021Cascaded Diffusion Models for High Fidelity Image Generationcited by 1,155
  19. 2021Variational Diffusion Modelscited by 1,145
  20. 2021Real-ESRGAN: Training Real-World Blind Super-Resolution with Pure Synthetic Datacited by 961
  21. 2015Accurate Image Super-Resolution Using Very Deep Convolutional Networkscited by 902
  22. 2020Pre-Trained Image Processing Transformercited by 891
  23. 2021Multi-Stage Progressive Image Restorationcited by 813
  24. 2020Image Quality Assessment: Unifying Structure and Texture Similaritycited by 792
  25. 2021Resolution-robust Large Mask Inpainting with Fourier Convolutionscited by 730
  26. 2016Deep Multi-scale Convolutional Neural Network for Dynamic Scene Deblurringcited by 722
  27. 2022Simple Baselines for Image Restorationcited by 701
  28. 2018Free-Form Image Inpainting with Gated Convolutioncited by 594
  29. 2018Generative Image Inpainting with Contextual Attentioncited by 578
  30. 2021Designing a Practical Degradation Model for Deep Blind Image Super-Resolutioncited by 524
  31. 2020Plug-and-Play Image Restoration with Deep Denoiser Priorcited by 467
  32. 2019Toward Real-World Single Image Super-Resolution: A New Benchmark and A New Modelcited by 400
  33. 2016Accelerating the Super-Resolution Convolutional Neural Networkcited by 393
  34. 2018Recovering Realistic Texture in Image Super-resolution by Deep Spatial Feature Transformcited by 369
  35. 2017DeblurGAN: Blind Motion Deblurring Using Conditional Adversarial Networkscited by 366
  36. 2019DeblurGAN-v2: Deblurring (Orders-of-Magnitude) Faster and Bettercited by 361
  37. 2018Scale-recurrent Network for Deep Image Deblurringcited by 332
  38. 2015Sketch-based Manga Retrieval using Manga109 Datasetcited by 326

50 most cited of 71 All 71 (JSON)

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2023: 23; 2024: 96; 2025: 98; 2026: 36.

253 papers.

Sort:

By area:

  1. 2024Diffusion Model-Based Image Editing: A Surveycited by 212
  2. 2024In-Context LoRA for Diffusion Transformerscited by 155
  3. 2023Upscale-A-Video: Temporal-Consistent Diffusion Model for Real-World Video Super-Resolutioncited by 131
  4. 2023Diffusion Models for Image Restoration and Enhancement: A Comprehensive Surveycited by 108
  5. 20243DGS-Enhancer: Enhancing Unbounded 3D Gaussian Splatting with View-consistent 2D Diffusion Priorscited by 106
  6. 2024Principled Probabilistic Imaging using Diffusion Models as Plug-and-Play Priorscited by 93
  7. 2024VmambaIR: Visual State Space Model for Image Restorationcited by 90
  8. 2023Hierarchical Integration Diffusion Model for Realistic Image Deblurringcited by 87
  9. 2024Arbitrary-steps Image Super-resolution via Diffusion Inversioncited by 85
  10. 2024Diffusion Models in Low-Level Vision: A Surveycited by 73
  11. 2023Reti-Diff: Illumination Degradation Image Restoration with Retinex-based Latent Diffusion Modelcited by 69
  12. 2024AddSR: Accelerating Diffusion-based Blind Super-Resolution with Adversarial Diffusion Distillationcited by 66
38 more
  1. 2024Efficient Diffusion Model for Image Restoration by Residual Shiftingcited by 64
  2. 2024Degradation-Guided One-Step Image Super-Resolution with Diffusion Priorscited by 64
  3. 2024Diffusion Models, Image Super-Resolution And Everything: A Surveycited by 63
  4. 2024Perceive-IR: Learning to Perceive Degradation Better for All-in-One Image Restorationcited by 60
  5. 2023EDiffSR: An Efficient Diffusion Probabilistic Model for Remote Sensing Image Super-Resolutioncited by 56
  6. 2024Boosting Diffusion Models with Moving Average Sampling in Frequency Domaincited by 53
  7. 2024Diff-Plugin: Revitalizing Details for Diffusion-based Low-level Taskscited by 47
  8. 2025UniRestore: Unified Perceptual and Task-Oriented Image Restoration Model Using Diffusion Priorcited by 39
  9. 2024FoundIR: Unleashing Million-scale Training Data to Advance Foundation Models for Image Restorationcited by 37
  10. 2025MB-TaylorFormer V2: Improved Multi-branch Linear Transformer Expanded by Taylor Formula for Image Restorationcited by 36
  11. 2024DiffIR2VR-Zero: Zero-Shot Video Restoration with Diffusion-based Image Restoration Modelscited by 33
  12. 2024PixWizard: Versatile Image-to-Image Visual Assistant with Open-Language Instructionscited by 31
  13. 2024Text-DiFuse: An Interactive Multi-Modal Image Fusion Framework based on Text-modulated Diffusion Modelcited by 31
  14. 2024Rethinking Diffusion Model for Multi-Contrast MRI Super-Resolutioncited by 29
  15. 2023CoDi: Conditional Diffusion Distillation for Higher-Fidelity and Faster Image Generationcited by 26
  16. 2024NTIRE 2024 Challenge on Low Light Image Enhancement: Methods and Resultscited by 26
  17. 2025Visual-Instructed Degradation Diffusion for All-in-One Image Restorationcited by 26
  18. 2023LLMGA: Multimodal Large Language Model based Generation Assistantcited by 25
  19. 2023DiffI2I: Efficient Diffusion Model for Image-to-Image Translationcited by 24
  20. 2024Teaching Tailored to Talent: Adverse Weather Restoration via Prompt Pool and Depth-Anything Constraintcited by 24
  21. 2024DiSR-NeRF: Diffusion-Guided View-Consistent Super-Resolution NeRFcited by 22
  22. 2023Super-NeRF: View-consistent Detail Generation for NeRF super-resolutioncited by 21
  23. 2023Towards Real-World Blind Face Restoration with Generative Diffusion Priorcited by 21
  24. 2025UnfoldIR: Rethinking Deep Unfolding Network in Illumination Degradation Image Restorationcited by 21
  25. 2024Building Bridges across Spatial and Temporal Resolutions: Reference-Based Super-Resolution via Change Priors and Conditional Diffusion Modelcited by 20
  26. 2024Motion-adaptive Separable Collaborative Filters for Blind Motion Deblurringcited by 20
  27. 2024PromptCIR: Blind Compressed Image Restoration with Prompt Learningcited by 20
  28. 2025UniUIR: Considering Underwater Image Restoration as An All-in-One Learnercited by 20
  29. 2023AdaDiff: Adaptive Step Selection for Fast Diffusion Modelscited by 18
  30. 2024IPAdapter-Instruct: Resolving Ambiguity in Image-based Conditioning using Instruct Promptscited by 18
  31. 2024LoRA-IR: Taming Low-Rank Experts for Efficient All-in-One Image Restorationcited by 18
  32. 2025UniDB: A Unified Diffusion Bridge Framework via Stochastic Optimal Controlcited by 18
  33. 2023Deep Equilibrium Diffusion Restoration with Parallel Samplingcited by 17
  34. 2024Beyond Image Super-Resolution for Image Recognition with Task-Driven Perceptual Losscited by 17
  35. 2024CascadedGaze: Efficiency in Global Context Extraction for Image Restorationcited by 16
  36. 2024Discrete to Continuous: Generating Smooth Transition Poses from Sign Language Observationcited by 16
  37. 2024MoE-DiffIR: Task-customized Diffusion Priors for Universal Compressed Image Restorationcited by 15
  38. 2024PromptRR: Diffusion Models as Prompt Generators for Single Image Reflection Removalcited by 14

50 most cited of 253 All 253 (JSON)

Similar, not cited

Close in wording, no citation either way

  1. 2023Denoising Diffusion Models for Plug-and-Play Image Restoration
  2. 2025Diffusion Image Prior
  3. 2023Restoration based Generative Models
  4. 2024Taming Diffusion Models for Image Restoration: A Review
  5. 2024DiffLoss: unleashing diffusion model as constraint for training image restoration network
  6. 2023DiffiT: Diffusion Vision Transformers for Image Generation
  7. 2025Diffusion Restoration Adapter for Real-World Image Restoration
  8. 2023Low-Light Image Enhancement with Wavelet-based Diffusion Models