mlsanityData to 7 Oct 2026

Pixel-level and Semantic-level Adjustable Super-resolution: A Dual-LoRA Approach

arXiv 2412.03017 ↗First posted 4 Dec 2024cs.CVcited by 93

On the map: Diffusion models → Inverse problems and image restoration

Abstract (arXiv)

Diffusion prior-based methods have shown impressive results in real-world image super-resolution (SR). However, most existing methods entangle pixel-level and semantic-level SR objectives in the training process, struggling to balance pixel-wise fidelity and perceptual quality. Meanwhile, users have varying preferences on SR results, thus it is demanded to develop an adjustable SR model that can be tailored to different fidelity-perception preferences during inference without re-training. We present Pixel-level and Semantic-level Adjustable SR (PiSA-SR), which learns two LoRA modules upon the pre-trained stable-diffusion (SD) model to achieve improved and adjustable SR results. We first formulate the SD-based SR problem as learning the residual between the low-quality input and the high-quality output, then show that the learning objective can be decoupled into two distinct LoRA weight spaces: one is characterized by the ℓ2-loss for pixel-level regression, and another is characterized by the LPIPS and classifier score distillation losses to extract semantic information from pre-trained classification and SD models. In its default setting, PiSA-SR can be performed in a single diffusion step, achieving leading real-world SR results in both quality and efficiency. By introducing two adjustable guidance scales on the two LoRA modules to control the strengths of pixel-wise fidelity and semantic-level details during inference, PiSASR can offer flexible SR results according to user preference without re-training. Codes and models can be found at https://github.com/csslc/PiSA-SR.

Key facts. Cited by 93 papers in the map (data through 7 Oct 2026). Placed in Inverse problems and image restoration, one of 14 directions in Diffusion models. Builds on 58 of its 73 references that are in the map. First posted on arXiv 4 Dec 2024.

Builds on

58 of 73 references.

Closest ancestors · 10

Same direction, newest first

  1. 2024DreamClear: High-Capacity Real-World Image Restoration with Privacy-Safe Dataset Curation24 Oct
  2. 2024One-Step Effective Diffusion Network for Real-World Image Super-Resolution12 Jun
  3. 2024XPSR: Cross-modal Priors for Diffusion-based Image Super-Resolution8 Mar
  4. 2023SeeSR: Towards Semantics-Aware Real-World Image Super-Resolution27 Nov
  5. 2023SinSR: Diffusion-Based Image Super-Resolution in a Single Step23 Nov
  6. 2023DiffBIR: Towards Blind Image Restoration with Generative Diffusion Prior29 Aug
  7. 2023Pixel-Aware Stable Diffusion for Realistic Image Super-resolution and Personalized Stylization28 Aug
  8. 2023ResShift: Efficient Diffusion Model for Image Super-resolution by Residual Shifting23 Jul
  9. 2023Exploiting Diffusion Prior for Real-World Image Super-Resolution11 May
  10. 2022Denoising Diffusion Restoration Models27 Jan

Other references · 48

  1. 2015Deep Residual Learning for Image Recognitioncited by 51,809
  2. 2020Denoising Diffusion Probabilistic Modelscited by 20,851
  3. 2021High-Resolution Image Synthesis with Latent Diffusion Modelscited by 18,694
  4. 2014Very Deep Convolutional Networks for Large-Scale Image Recognitioncited by 15,267
  5. 2021LoRA: Low-Rank Adaptation of Large Language Modelscited by 14,084
  6. 2018The Unreasonable Effectiveness of Deep Features as a Perceptual Metriccited by 11,029
  7. 2020Score-Based Generative Modeling through Stochastic Differential Equationscited by 8,964
  8. 2018A Style-Based Generator Architecture for Generative Adversarial Networkscited by 5,986
  9. 2022Classifier-Free Diffusion Guidancecited by 5,717
  10. 2023Adding Conditional Control to Text-to-Image Diffusion Modelscited by 5,466
  11. 2016Perceptual Losses for Real-Time Style Transfer and Super-Resolutioncited by 2,678
  12. 2016Photo-Realistic Single Image Super-Resolution Using a Generative Adversarial Networkcited by 1,898
36 more
  1. 2021SwinIR: Image Restoration Using Swin Transformercited by 1,869
  2. 2016Beyond a Gaussian Denoiser: Residual Learning of Deep CNN for Image Denoisingcited by 1,352
  3. 2023T2I-Adapter: Learning Adapters to Dig out More Controllable Ability for Text-to-Image Diffusion Modelscited by 1,339
  4. 2018ESRGAN: Enhanced Super-Resolution Generative Adversarial Networkscited by 1,250
  5. 2023ProlificDreamer: High-Fidelity and Diverse Text-to-3D Generation with Variational Score Distillationcited by 1,207
  6. 2021MUSIQ: Multi-scale Image Quality Transformercited by 1,058
  7. 2023One-step Diffusion with Distribution Matching Distillationcited by 978
  8. 2022Exploring CLIP for Assessing the Look and Feel of Imagescited by 974
  9. 2021Real-ESRGAN: Training Real-World Blind Super-Resolution with Pure Synthetic Datacited by 961
  10. 2018Residual Dense Network for Image Super-Resolutioncited by 796
  11. 2020Image Quality Assessment: Unifying Structure and Texture Similaritycited by 792
  12. 2017The Perception-Distortion Tradeoffcited by 582
  13. 2021Designing a Practical Degradation Model for Deep Blind Image Super-Resolutioncited by 524
  14. 2022MANIQA: Multi-dimension Attention Network for No-Reference Image Quality Assessmentcited by 514
  15. 2022Activating More Pixels in Image Super-Resolution Transformercited by 433
  16. 2019Toward Real-World Single Image Super-Resolution: A New Benchmark and A New Modelcited by 400
  17. 2019Deep Learning for Image Super-resolution: A Surveycited by 368
  18. 2015Deeply-Recursive Convolutional Network for Image Super-Resolutioncited by 317
  19. 2023Recognize Anything: A Strong Image Tagging Modelcited by 305
  20. 2024Scaling Up to Excellence: Practicing Model Scaling for Photo-Realistic Image Restoration In the Wildcited by 281
  21. 2020Component Divide-and-Conquer for Real-World Image Super-Resolutioncited by 243
  22. 2018Deep Back-Projection Networks For Super-Resolutioncited by 224
  23. 2023ZipLoRA: Any Subject in Any Style by Effectively Merging LoRAscited by 194
  24. 2022Efficient Long-Range Attention Network for Image Super-resolutioncited by 151
  25. 2023Text-to-3D with Classifier Score Distillationcited by 127
  26. 2022Details or Artifacts: A Locally Discriminative Learning Approach to Realistic Image Super-Resolutioncited by 114
  27. 2022Efficient and Degradation-Adaptive Network for Real-World Image Super-Resolutioncited by 96
  28. 2024Adversarial Diffusion Compression for Real-World Image Super-Resolutioncited by 49
  29. 2018Deep Network Interpolation for Continuous Imagery Effect Transitioncited by 47
  30. 2024Customizing Text-to-Image Models with a Single Image Paircited by 36
  31. 2022MM-RealSR: Metric Learning based Interactive Modulation for Real-World Super-Resolutioncited by 33
  32. 2021Best-Buddy GANs for Highly Detailed Image Super-Resolutioncited by 30
  33. 2024ScaleDreamer: Scalable Text-to-3D Synthesis with Asynchronous Score Distillationcited by 27
  34. 2022Perception-Distortion Balanced ADMM Optimization for Single-Image Super-Resolutioncited by 19
  35. 2023Perception-Distortion Balanced Super-Resolution: A Multi-Objective Optimization Perspectivecited by 11
  36. 2024CTRLorALTer: Conditional LoRAdapter for Efficient 0-Shot Control & Altering of T2I Modelscited by 4

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2025: 41; 2026: 52.

93 papers.

Sort:

By area:

  1. 2025DiT4SR: Taming Diffusion Transformer for Real-World Image Super-Resolutioncited by 50
  2. 2025Toward Generalized Image Quality Assessment: Relaxing the Perfect Reference Quality Assumptioncited by 34
  3. 2025Perceive, Understand and Restore: Real-World Image Super-Resolution with Autoregressive Multimodal Generative Modelscited by 24
  4. 2025Fine-structure Preserved Real-world Image Super-resolution via Transfer VAE Trainingcited by 21
  5. 2025NTIRE 2025 Challenge on Short-form UGC Video Quality Assessment and Enhancement: Methods and Resultscited by 14
  6. 2025One-Step Diffusion for Detail-Rich and Temporally Consistent Video Super-Resolutioncited by 14
  7. 2025OMGSR: You Only Need One Mid-timestep Guidance for Real-World Image Super-Resolutioncited by 14
  8. 2025NTIRE 2025 Challenge on Real-World Face Restoration: Methods and Resultscited by 10
  9. 2025PocketSR: The Super-Resolution Expert in Your Pocket Mobilescited by 8
  10. 2025Progressive Rendering Distillation: Adapting Stable Diffusion for Instant Text-to-Mesh Generation without 3D Datacited by 7
  11. 2025One-Step Diffusion Transformer for Controllable Real-World Image Super-Resolutioncited by 7
  12. 2025Stream-DiffVSR: Low-Latency Streamable Video Super-Resolution via Auto-Regressive Diffusioncited by 7
38 more
  1. 2025Time-Aware One Step Diffusion Network for Real-World Image Super-Resolutioncited by 6
  2. 2026The First Challenge on Mobile Real-World Image Super-Resolution at NTIRE 2026: Benchmark Results and Method Overviewcited by 6
  3. 2025RealSR-R1: Reinforcement Learning for Real-World Image Super-Resolution with Vision-Language Chain-of-Thoughtcited by 5
  4. 2025Hallucination Score: Towards Mitigating Hallucinations in Generative Image Super-Resolutioncited by 5
  5. 2026Trust but Verify: Adaptive Conditioning for Reference-Based Diffusion Super-Resolution via Implicit Reference Correlation Modelingcited by 5
  6. 2026Bilevel Layer-Positioning LoRA for Real Image Dehazingcited by 5
  7. 2026SparkVSR: Interactive Video Super-Resolution via Sparse Keyframe Propagationcited by 5
  8. 2025RIFLE: Removal of Image Flicker-Banding via Latent Diffusion Enhancementcited by 4
  9. 2026Polarization Uncertainty-Guided Diffusion Model for Color Polarization Image Demosaickingcited by 4
  10. 2026Improved Adversarial Diffusion Compression for Real-World Video Super-Resolutioncited by 4
  11. 2026Can Nano Banana 2 Replace Traditional Image Restoration Models? An Evaluation of Its Performance on Image Restoration Taskscited by 4
  12. 2025Bridging the Perception Gap in Image Super-Resolution Evaluationcited by 3
  13. 2025EchoGen: Generating Visual Echoes in Any Scene via Feed-Forward Subject-Driven Auto-Regressive Modelcited by 3
  14. 2025Bridging Fidelity-Reality with Controllable One-Step Diffusion for Image Super-Resolutioncited by 3
  15. 2025FinPercep-RM: A Fine-grained Reward Model and Co-evolutionary Curriculum for RL-based Real-world Super-Resolutioncited by 3
  16. 2026FiDeSR: High-Fidelity and Detail-Preserving One-Step Diffusion Super-Resolutioncited by 3
  17. 2026GDPO-SR: Group Direct Preference Optimization for One-Step Generative Image Super-Resolutioncited by 3
  18. 2025One-Step Residual Shifting Diffusion for Image Super-Resolution via Distillationcited by 2
  19. 2025CTSR: Controllable Fidelity-Realness Trade-off Distillation for Real-World Image Super Resolutioncited by 2
  20. 2025Enhanced Semantic Extraction and Guidance for UGC Image Super Resolutioncited by 2
  21. 2025DP2O-SR: Direct Perceptual Preference Optimization for Real-World Image Super-Resolutioncited by 2
  22. 2025Enhancing Diffusion-based Restoration Models via Difficulty-Adaptive Reinforcement Learning with IQA Rewardcited by 2
  23. 2025FRAMER: Frequency-Aligned Self-Distillation with Adaptive Modulation Leveraging Diffusion Priors for Real-World Image Super-Resolutioncited by 2
  24. 2026Combined Flicker-banding and Moire Removal for Screen-Captured Imagescited by 2
  25. 2026Joint Geometric and Trajectory Consistency Learning for One-Step Real-World Super-Resolutioncited by 2
  26. 2026How far have we gone in Generative Image Restoration? A study on its capability, limitations and evaluation practicescited by 2
  27. 2026VOSR: A Vision-Only Generative Model for Image Super-Resolutioncited by 2
  28. 2025InstructRestore: Region-Customized Image Restoration with Human Instructionscited by 1
  29. 2025HonestFace: Towards Honest Face Restoration with One-Step Diffusion Modelcited by 1
  30. 2025Low-Resolution Editing is All You Need for High-Resolution Editingcited by 1
  31. 2025OmniScaleSR: Unleashing Scale-Controlled Diffusion Prior for Faithful and Realistic Arbitrary-Scale Image Super-Resolutioncited by 1
  32. 2025F2IDiff: Real-world Image Super-resolution using Feature to Image Diffusion Foundation Modelcited by 1
  33. 2026Taming Real-World Space-Time Video Super-Resolution with One-Step Diffusioncited by 1
  34. 2026Bird-SR: Bidirectional Reward-Guided Diffusion for Real-World Image Super-Resolutioncited by 1
  35. 2026CompSplat: Compression-aware 3D Gaussian Splatting for Real-world Videocited by 1
  36. 2026Restoration Adaptation for Semantic Segmentation on Low Quality Imagescited by 1
  37. 2026Spectral and Trajectory Regularization for Diffusion Transformer Super-Resolutioncited by 1
  38. 2026CustomTex: High-fidelity Indoor Scene Texturing via Multi-Reference Customizationcited by 1

50 most cited of 93 All 93 (JSON)

Similar, not cited

Close in wording, no citation either way

  1. 2024Improving Consistency in Diffusion Models for Image Super-Resolution
  2. 2024Degradation-Guided One-Step Image Super-Resolution with Diffusion Priors
  3. 2025Semantic-Guided Diffusion Model for Single-Step Image Super-Resolution
  4. 2026Revisiting the Perception-Distortion Trade-off with Spatial-Semantic Guided Super-Resolution
  5. 2024Hero-SR: One-Step Diffusion for Super-Resolution with Human Perception Priors
  6. 2024ControlSR: Taming Diffusion Models for Consistent Real-World Image Super Resolution
  7. 2026Disentangled Textual Priors for Diffusion-based Image Super-Resolution
  8. 2025Leveraging Vision-Language Models to Select Trustworthy Super-Resolution Samples Generated by Diffusion Models