# Pixel-level and Semantic-level Adjustable Super-resolution: A Dual-LoRA Approach

arXiv: https://arxiv.org/abs/2412.03017 · HTML: https://mlsanity.com/paper/2412.03017/

## Key facts

- Cited by 97 papers in the map (data through 14 Oct 2026).
- Placed in Inverse problems and image restoration, one of 14 directions in Diffusion models.
- Builds on 58 of its 73 references that are in the map.
- First posted on arXiv 4 Dec 2024.

## Placed in

- Diffusion models / [Inverse problems and image restoration](https://mlsanity.com/diffusion/inverse-problems-and-image-restoration/)

## Builds on (most cited first, 30 of 58)

- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [Denoising Diffusion Probabilistic Models](https://mlsanity.com/paper/2006.11239/) (2020)
- [High-Resolution Image Synthesis with Latent Diffusion Models](https://mlsanity.com/paper/2112.10752/) (2021)
- [Very Deep Convolutional Networks for Large-Scale Image Recognition](https://mlsanity.com/paper/1409.1556/) (2014)
- [LoRA: Low-Rank Adaptation of Large Language Models](https://mlsanity.com/paper/2106.09685/) (2021)
- [The Unreasonable Effectiveness of Deep Features as a Perceptual Metric](https://mlsanity.com/paper/1801.03924/) (2018)
- [Score-Based Generative Modeling through Stochastic Differential Equations](https://mlsanity.com/paper/2011.13456/) (2020)
- [A Style-Based Generator Architecture for Generative Adversarial Networks](https://mlsanity.com/paper/1812.04948/) (2018)
- [Classifier-Free Diffusion Guidance](https://mlsanity.com/paper/2207.12598/) (2022)
- [Adding Conditional Control to Text-to-Image Diffusion Models](https://mlsanity.com/paper/2302.05543/) (2023)
- [Perceptual Losses for Real-Time Style Transfer and Super-Resolution](https://mlsanity.com/paper/1603.08155/) (2016)
- [Photo-Realistic Single Image Super-Resolution Using a Generative Adversarial Network](https://mlsanity.com/paper/1609.04802/) (2016)
- [SwinIR: Image Restoration Using Swin Transformer](https://mlsanity.com/paper/2108.10257/) (2021)
- [Beyond a Gaussian Denoiser: Residual Learning of Deep CNN for Image Denoising](https://mlsanity.com/paper/1608.03981/) (2016)
- [T2I-Adapter: Learning Adapters to Dig out More Controllable Ability for Text-to-Image Diffusion Models](https://mlsanity.com/paper/2302.08453/) (2023)
- [ESRGAN: Enhanced Super-Resolution Generative Adversarial Networks](https://mlsanity.com/paper/1809.00219/) (2018)
- [ProlificDreamer: High-Fidelity and Diverse Text-to-3D Generation with Variational Score Distillation](https://mlsanity.com/paper/2305.16213/) (2023)
- [MUSIQ: Multi-scale Image Quality Transformer](https://mlsanity.com/paper/2108.05997/) (2021)
- [One-step Diffusion with Distribution Matching Distillation](https://mlsanity.com/paper/2311.18828/) (2023)
- [Exploring CLIP for Assessing the Look and Feel of Images](https://mlsanity.com/paper/2207.12396/) (2022)
- [Real-ESRGAN: Training Real-World Blind Super-Resolution with Pure Synthetic Data](https://mlsanity.com/paper/2107.10833/) (2021)
- [Denoising Diffusion Restoration Models](https://mlsanity.com/paper/2201.11793/) (2022)
- [Image Quality Assessment: Unifying Structure and Texture Similarity](https://mlsanity.com/paper/2004.07728/) (2020)
- [Residual Dense Network for Image Super-Resolution](https://mlsanity.com/paper/1802.08797/) (2018)
- [The Perception-Distortion Tradeoff](https://mlsanity.com/paper/1711.06077/) (2017)
- [Designing a Practical Degradation Model for Deep Blind Image Super-Resolution](https://mlsanity.com/paper/2103.14006/) (2021)
- [MANIQA: Multi-dimension Attention Network for No-Reference Image Quality Assessment](https://mlsanity.com/paper/2204.08958/) (2022)
- [Exploiting Diffusion Prior for Real-World Image Super-Resolution](https://mlsanity.com/paper/2305.07015/) (2023)
- [Activating More Pixels in Image Super-Resolution Transformer](https://mlsanity.com/paper/2205.04437/) (2022)
- [Toward Real-World Single Image Super-Resolution: A New Benchmark and A New Model](https://mlsanity.com/paper/1904.00523/) (2019)

## Built on by (most cited first, 30 of 97)

- [DiT4SR: Taming Diffusion Transformer for Real-World Image Super-Resolution](https://mlsanity.com/paper/2503.23580/) (2025)
- [Toward Generalized Image Quality Assessment: Relaxing the Perfect Reference Quality Assumption](https://mlsanity.com/paper/2503.11221/) (2025)
- [Perceive, Understand and Restore: Real-World Image Super-Resolution with Autoregressive Multimodal Generative Models](https://mlsanity.com/paper/2503.11073/) (2025)
- [Fine-structure Preserved Real-world Image Super-resolution via Transfer VAE Training](https://mlsanity.com/paper/2507.20291/) (2025)
- [One-Step Diffusion for Detail-Rich and Temporally Consistent Video Super-Resolution](https://mlsanity.com/paper/2506.15591/) (2025)
- [OMGSR: You Only Need One Mid-timestep Guidance for Real-World Image Super-Resolution](https://mlsanity.com/paper/2508.08227/) (2025)
- [NTIRE 2025 Challenge on Short-form UGC Video Quality Assessment and Enhancement: Methods and Results](https://mlsanity.com/paper/2504.13131/) (2025)
- [NTIRE 2025 Challenge on Real-World Face Restoration: Methods and Results](https://mlsanity.com/paper/2504.14600/) (2025)
- [PocketSR: The Super-Resolution Expert in Your Pocket Mobiles](https://mlsanity.com/paper/2510.03012/) (2025)
- [One-Step Diffusion Transformer for Controllable Real-World Image Super-Resolution](https://mlsanity.com/paper/2511.17138/) (2025)
- [Progressive Rendering Distillation: Adapting Stable Diffusion for Instant Text-to-Mesh Generation without 3D Data](https://mlsanity.com/paper/2503.21694/) (2025)
- [Time-Aware One Step Diffusion Network for Real-World Image Super-Resolution](https://mlsanity.com/paper/2508.16557/) (2025)
- [Stream-DiffVSR: Low-Latency Streamable Video Super-Resolution via Auto-Regressive Diffusion](https://mlsanity.com/paper/2512.23709/) (2025)
- [Hallucination Score: Towards Mitigating Hallucinations in Generative Image Super-Resolution](https://mlsanity.com/paper/2507.14367/) (2025)
- [Bilevel Layer-Positioning LoRA for Real Image Dehazing](https://mlsanity.com/paper/2603.10872/) (2026)
- [The First Challenge on Mobile Real-World Image Super-Resolution at NTIRE 2026: Benchmark Results and Method Overview](https://mlsanity.com/paper/2604.17306/) (2026)
- [RealSR-R1: Reinforcement Learning for Real-World Image Super-Resolution with Vision-Language Chain-of-Thought](https://mlsanity.com/paper/2506.16796/) (2025)
- [Trust but Verify: Adaptive Conditioning for Reference-Based Diffusion Super-Resolution via Implicit Reference Correlation Modeling](https://mlsanity.com/paper/2602.01864/) (2026)
- [Improved Adversarial Diffusion Compression for Real-World Video Super-Resolution](https://mlsanity.com/paper/2603.00458/) (2026)
- [SparkVSR: Interactive Video Super-Resolution via Sparse Keyframe Propagation](https://mlsanity.com/paper/2603.16864/) (2026)
- [VOSR: A Vision-Only Generative Model for Image Super-Resolution](https://mlsanity.com/paper/2604.03225/) (2026)
- [Bridging the Perception Gap in Image Super-Resolution Evaluation](https://mlsanity.com/paper/2503.13074/) (2025)
- [RIFLE: Removal of Image Flicker-Banding via Latent Diffusion Enhancement](https://mlsanity.com/paper/2509.24644/) (2025)
- [Polarization Uncertainty-Guided Diffusion Model for Color Polarization Image Demosaicking](https://mlsanity.com/paper/2602.23847/) (2026)
- [FiDeSR: High-Fidelity and Detail-Preserving One-Step Diffusion Super-Resolution](https://mlsanity.com/paper/2603.02692/) (2026)
- [GDPO-SR: Group Direct Preference Optimization for One-Step Generative Image Super-Resolution](https://mlsanity.com/paper/2603.16769/) (2026)
- [Can Nano Banana 2 Replace Traditional Image Restoration Models? An Evaluation of Its Performance on Image Restoration Tasks](https://mlsanity.com/paper/2604.03061/) (2026)
- [One-Step Residual Shifting Diffusion for Image Super-Resolution via Distillation](https://mlsanity.com/paper/2503.13358/) (2025)
- [EchoGen: Generating Visual Echoes in Any Scene via Feed-Forward Subject-Driven Auto-Regressive Model](https://mlsanity.com/paper/2509.26127/) (2025)
- [Bridging Fidelity-Reality with Controllable One-Step Diffusion for Image Super-Resolution](https://mlsanity.com/paper/2512.14061/) (2025)

## Abstract (arXiv)

Diffusion prior-based methods have shown impressive results in real-world image super-resolution (SR). However, most existing methods entangle pixel-level and semantic-level SR objectives in the training process, struggling to balance pixel-wise fidelity and perceptual quality. Meanwhile, users have varying preferences on SR results, thus it is demanded to develop an adjustable SR model that can be tailored to different fidelity-perception preferences during inference without re-training. We present Pixel-level and Semantic-level Adjustable SR (PiSA-SR), which learns two LoRA modules upon the pre-trained stable-diffusion (SD) model to achieve improved and adjustable SR results. We first formulate the SD-based SR problem as learning the residual between the low-quality input and the high-quality output, then show that the learning objective can be decoupled into two distinct LoRA weight spaces: one is characterized by the ℓ2-loss for pixel-level regression, and another is characterized by the LPIPS and classifier score distillation losses to extract semantic information from pre-trained classification and SD models. In its default setting, PiSA-SR can be performed in a single diffusion step, achieving leading real-world SR results in both quality and efficiency. By introducing two adjustable guidance scales on the two LoRA modules to control the strengths of pixel-wise fidelity and semantic-level details during inference, PiSASR can offer flexible SR results according to user preference without re-training. Codes and models can be found at https://github.com/csslc/PiSA-SR.