mlsanityData to 7 Oct 2026

Efficient Image Restoration through Low-Rank Adaptation and Stable Diffusion XL

arXiv 2408.17060 ↗First posted 30 Aug 2024cs.CVcited by 0

On the map: Diffusion models → Inverse problems and image restoration

Abstract (arXiv)

In this study, we propose an enhanced image restoration model, SUPIR, based on the integration of two low-rank adaptive (LoRA) modules with the Stable Diffusion XL (SDXL) framework. Our method leverages the advantages of LoRA to fine-tune SDXL models, thereby significantly improving image restoration quality and efficiency. We collect 2600 high-quality real-world images, each with detailed descriptive text, for training the model. The proposed method is evaluated on standard benchmarks and achieves excellent performance, demonstrated by higher peak signal-to-noise ratio (PSNR), lower learned perceptual image patch similarity (LPIPS), and higher structural similarity index measurement (SSIM) scores. These results underscore the effectiveness of combining LoRA with SDXL for advanced image restoration tasks, highlighting the potential of our approach in generating high-fidelity restored images.

Key facts. No citing papers in the map yet (data through 7 Oct 2026). Placed in Inverse problems and image restoration, one of 14 directions in Diffusion models. Builds on 43 of its 48 references that are in the map. First posted on arXiv 30 Aug 2024.

Builds on

43 of 48 references.

Closest ancestors · 9

Same direction, newest first

  1. 2023DiffBIR: Towards Blind Image Restoration with Generative Diffusion Prior29 Aug
  2. 2023Pixel-Aware Stable Diffusion for Realistic Image Super-resolution and Personalized Stylization28 Aug
  3. 2023Hierarchical Integration Diffusion Model for Realistic Image Deblurring22 May
  4. 2023Denoising Diffusion Models for Plug-and-Play Image Restoration15 May
  5. 2023Exploiting Diffusion Prior for Real-World Image Super-Resolution11 May
  6. 2023DiffIR: Efficient Diffusion Model for Image Restoration16 Mar
  7. 2022Multiscale Structure Guided Diffusion for Image Deblurring4 Dec
  8. 2022Zero-Shot Image Restoration Using Denoising Diffusion Null-Space Model1 Dec
  9. 2022Denoising Diffusion Restoration Models27 Jan

Other references · 34

  1. 2017Attention Is All You Needcited by 59,932
  2. 2020An Image is Worth 16x16 Words: Transformers for Image Recognition at Scalecited by 29,915
  3. 2020Denoising Diffusion Probabilistic Modelscited by 20,851
  4. 2015U-Net: Convolutional Networks for Biomedical Image Segmentationcited by 19,604
  5. 2021High-Resolution Image Synthesis with Latent Diffusion Modelscited by 18,694
  6. 2021LoRA: Low-Rank Adaptation of Large Language Modelscited by 14,084
  7. 2018The Unreasonable Effectiveness of Deep Features as a Perceptual Metriccited by 11,029
  8. 2020Denoising Diffusion Implicit Modelscited by 9,655
  9. 2023Visual Instruction Tuningcited by 7,792
  10. 2016Densely Connected Convolutional Networkscited by 6,691
  11. 2022Photorealistic Text-to-Image Diffusion Models with Deep Language Understandingcited by 6,322
  12. 2022Classifier-Free Diffusion Guidancecited by 5,717
22 more
  1. 2023Adding Conditional Control to Text-to-Image Diffusion Modelscited by 5,466
  2. 2023SDXL: Improving Latent Diffusion Models for High-Resolution Image Synthesiscited by 4,290
  3. 2021Zero-Shot Text-to-Image Generationcited by 4,276
  4. 2023Stable Video Diffusion: Scaling Latent Video Diffusion Models to Large Datasetscited by 2,526
  5. 2023T2I-Adapter: Learning Adapters to Dig out More Controllable Ability for Text-to-Image Diffusion Modelscited by 1,339
  6. 2014Image Super-Resolution Using Deep Convolutional Networkscited by 1,218
  7. 2022Diffusion Models: A Comprehensive Survey of Methods and Applicationscited by 1,107
  8. 2022Diffusion Models in Vision: A Surveycited by 1,008
  9. 2022Tune-A-Video: One-Shot Tuning of Image Diffusion Models for Text-to-Video Generationcited by 958
  10. 2023Adversarial Diffusion Distillationcited by 741
  11. 2017Learning Deep CNN Denoiser Prior for Image Restorationcited by 407
  12. 2018Recovering Realistic Texture in Image Super-resolution by Deep Spatial Feature Transformcited by 369
  13. 2024LoRA Learns Less and Forgets Lesscited by 313
  14. 2023Show-1: Marrying Pixel and Latent Diffusion Models for Text-to-Video Generationcited by 300
  15. 2024Scaling Up to Excellence: Practicing Model Scaling for Photo-Realistic Image Restoration In the Wildcited by 281
  16. 2024SDXL-Lightning: Progressive Adversarial Diffusion Distillationcited by 258
  17. 2020PIPAL: a Large-Scale Image Quality Assessment Dataset for Perceptual Image Restorationcited by 177
  18. 2022Deep Image Deblurring: A Surveycited by 94
  19. 2022Accurate Image Restoration with Attention Retractable Transformercited by 85
  20. 2022Large-scale Text-to-Image Generation Models for Visual Artists' Creative Workscited by 60
  21. 2020Neural Sparse Representation for Image Restorationcited by 22
  22. 2017Building Fast and Compact Convolutional Neural Networks for Offline Handwritten Chinese Character Recognitioncited by 14

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2025Diffusion Once and Done: Degradation-Aware LoRA for Efficient All-in-One Image Restoration
  2. 2024Training-Free Large Model Priors for Multiple-in-One Image Restoration
  3. 2024LoRA-IR: Taming Low-Rank Experts for Efficient All-in-One Image Restoration
  4. 2023Low-Light Image Enhancement with Wavelet-based Diffusion Models
  5. 2023A Unified Conditional Framework for Diffusion-based Image Restoration
  6. 2022ADIR: Adaptive Diffusion for Image Reconstruction
  7. 2023LoRA-Enhanced Distillation on Guided Diffusion Models
  8. 2023Synthesizing Realistic Image Restoration Training Pairs: A Diffusion Approach