mlsanityData to 7 Oct 2026

XPSR: Cross-modal Priors for Diffusion-based Image Super-Resolution

arXiv 2403.05049 ↗First posted 8 Mar 2024cs.CVcited by 41

On the map: Diffusion models → Inverse problems and image restoration

Abstract (arXiv)

Diffusion-based methods, endowed with a formidable generative prior, have received increasing attention in Image Super-Resolution (ISR) recently. However, as low-resolution (LR) images often undergo severe degradation, it is challenging for ISR models to perceive the semantic and degradation information, resulting in restoration images with incorrect content or unrealistic artifacts. To address these issues, we propose a textitCross-modal Priors for Super-Resolution (XPSR) framework. Within XPSR, to acquire precise and comprehensive semantic conditions for the diffusion model, cutting-edge Multimodal Large Language Models (MLLMs) are utilized. To facilitate better fusion of cross-modal priors, a textitSemantic-Fusion Attention is raised. To distill semantic-preserved information instead of undesired degradations, a textitDegradation-Free Constraint is attached between LR and its high-resolution (HR) counterpart. Quantitative and qualitative results show that XPSR is capable of generating high-fidelity and high-realism images across synthetic and real-world datasets. Codes are released at urlhttps://github.com/qyp2000/XPSR.

Key facts. Cited by 41 papers in the map (data through 7 Oct 2026). Placed in Inverse problems and image restoration, one of 14 directions in Diffusion models. Builds on 65 of its 82 references that are in the map. First posted on arXiv 8 Mar 2024.

Builds on

65 of 82 references.

Closest ancestors · 7

Same direction, newest first

  1. 2023SeeSR: Towards Semantics-Aware Real-World Image Super-Resolution27 Nov
  2. 2023DiffBIR: Towards Blind Image Restoration with Generative Diffusion Prior29 Aug
  3. 2023Pixel-Aware Stable Diffusion for Realistic Image Super-resolution and Personalized Stylization28 Aug
  4. 2023Exploiting Diffusion Prior for Real-World Image Super-Resolution11 May
  5. 2023Generative Diffusion Prior for Unified Image Restoration and Enhancement3 Apr
  6. 2023Denoising Diffusion Probabilistic Models for Robust Image Super-Resolution in the Wild15 Feb
  7. 2022Denoising Diffusion Restoration Models27 Jan

Other references · 58

  1. 2015Deep Residual Learning for Image Recognitioncited by 51,809
  2. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,456
  3. 2020Denoising Diffusion Probabilistic Modelscited by 20,851
  4. 2021High-Resolution Image Synthesis with Latent Diffusion Modelscited by 18,694
  5. 2023GPT-4 Technical Reportcited by 18,132
  6. 2023LLaMA: Open and Efficient Foundation Language Modelscited by 13,958
  7. 2018The Unreasonable Effectiveness of Deep Features as a Perceptual Metriccited by 11,029
  8. 2023Visual Instruction Tuningcited by 7,792
  9. 2022Hierarchical Text-Conditional Image Generation with CLIP Latentscited by 6,539
  10. 2022Photorealistic Text-to-Image Diffusion Models with Deep Language Understandingcited by 6,322
  11. 2023BLIP-2: Bootstrapping Language-Image Pre-training with Frozen Image Encoders and Large Language Modelscited by 6,069
  12. 2018A Style-Based Generator Architecture for Generative Adversarial Networkscited by 5,986
38 more
  1. 2022Classifier-Free Diffusion Guidancecited by 5,717
  2. 2023Adding Conditional Control to Text-to-Image Diffusion Modelscited by 5,466
  3. 2022BLIP: Bootstrapping Language-Image Pre-training for Unified Vision-Language Understanding and Generationcited by 4,310
  4. 2021Improved Denoising Diffusion Probabilistic Modelscited by 3,736
  5. 2023MiniGPT-4: Enhancing Vision-Language Understanding with Advanced Large Language Modelscited by 2,689
  6. 2022InstructPix2Pix: Learning to Follow Image Editing Instructionscited by 2,591
  7. 2022Prompt-to-Prompt Image Editing with Cross Attention Controlcited by 2,409
  8. 2021SwinIR: Image Restoration Using Swin Transformercited by 1,869
  9. 2017Decoupled Weight Decay Regularizationcited by 1,722
  10. 2017Enhanced Deep Residual Networks for Single Image Super-Resolutioncited by 1,543
  11. 2021Image Super-Resolution via Iterative Refinementcited by 1,506
  12. 2023T2I-Adapter: Learning Adapters to Dig out More Controllable Ability for Text-to-Image Diffusion Modelscited by 1,339
  13. 2018Image Super-Resolution Using Very Deep Residual Channel Attention Networkscited by 1,256
  14. 2014Image Super-Resolution Using Deep Convolutional Networkscited by 1,218
  15. 2022Imagic: Text-Based Real Image Editing with Diffusion Modelscited by 1,172
  16. 2021MUSIQ: Multi-scale Image Quality Transformercited by 1,058
  17. 2022Exploring CLIP for Assessing the Look and Feel of Imagescited by 974
  18. 2021Real-ESRGAN: Training Real-World Blind Super-Resolution with Pure Synthetic Datacited by 961
  19. 2023A Survey on Multimodal Large Language Modelscited by 898
  20. 2020Image Quality Assessment: Unifying Structure and Texture Similaritycited by 792
  21. 2023LLaMA-Adapter V2: Parameter-Efficient Visual Instruction Modelcited by 602
  22. 2021Designing a Practical Degradation Model for Deep Blind Image Super-Resolutioncited by 524
  23. 2022MANIQA: Multi-dimension Attention Network for No-Reference Image Quality Assessmentcited by 514
  24. 2019Toward Real-World Single Image Super-Resolution: A New Benchmark and A New Modelcited by 400
  25. 2016Accelerating the Super-Resolution Convolutional Neural Networkcited by 393
  26. 2018Recovering Realistic Texture in Image Super-resolution by Deep Spatial Feature Transformcited by 369
  27. 2020PULSE: Self-Supervised Photo Upsampling via Latent Space Exploration of Generative Modelscited by 358
  28. 2019KonIQ-10k: An ecologically valid database for deep learning of blind image quality assessmentcited by 351
  29. 2023Recognize Anything: A Strong Image Tagging Modelcited by 305
  30. 2023InternLM-XComposer: A Vision-Language Large Model for Advanced Text-image Comprehension and Compositioncited by 297
  31. 2023Q-Bench: A Benchmark for General-Purpose Foundation Models on Low-level Visioncited by 247
  32. 2020Component Divide-and-Conquer for Real-World Image Super-Resolutioncited by 243
  33. 2017Learning a Single Convolutional Super-Resolution Network for Multiple Degradationscited by 227
  34. 2019Blind Super-Resolution With Iterative Kernel Correctioncited by 203
  35. 2019Image Processing Using Multi-Code GAN Priorcited by 185
  36. 2020GLEAN: Generative Latent Bank for Large-Factor Image Super-Resolutioncited by 158
  37. 2023Black-Box Prompt Optimization: Aligning Large Language Models without Model Trainingcited by 122
  38. 2022Details or Artifacts: A Locally Discriminative Learning Approach to Realistic Image Super-Resolutioncited by 114

50 most cited of 58 All 58 (JSON)

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2024: 9; 2025: 21; 2026: 11.

41 papers.

Sort:

By area:

  1. 2024Pixel-level and Semantic-level Adjustable Super-resolution: A Dual-LoRA Approachcited by 93
  2. 2025DiT4SR: Taming Diffusion Transformer for Real-World Image Super-Resolutioncited by 50
  3. 2024Adversarial Diffusion Compression for Real-World Image Super-Resolutioncited by 49
  4. 2025FlashVSR: Towards Real-Time Diffusion-Based Streaming Video Super-Resolutioncited by 40
  5. 2024FaithDiff: Unleashing Diffusion Priors for Faithful Image Super-resolutioncited by 31
  6. 2025Fine-structure Preserved Real-world Image Super-resolution via Transfer VAE Trainingcited by 21
  7. 2025The Power of Context: How Multimodality Improves Image Super-Resolutioncited by 15
  8. 2025NTIRE 2025 Challenge on Short-form UGC Video Quality Assessment and Enhancement: Methods and Resultscited by 14
  9. 2025One-Step Diffusion for Detail-Rich and Temporally Consistent Video Super-Resolutioncited by 14
  10. 2025NTIRE 2025 Challenge on Short-form UGC Video Quality Assessment and Enhancement: KwaiSR Dataset and Studycited by 11
  11. 2024HoliSDiP: Image Super-Resolution via Holistic Semantics and Diffusion Priorcited by 8
  12. 2025Acquire and then Adapt: Squeezing out Text-to-Image Model for Image Restorationcited by 8
29 more
  1. 2024RFSR: Improving ISR Diffusion Models via Reward Feedback Learningcited by 5
  2. 2025UniRes: Universal Image Restoration for Complex Degradationscited by 5
  3. 2024A New Dataset and Framework for Real-World Blurred Images Super-Resolutioncited by 3
  4. 2024OAPT: Offset-Aware Partition Transformer for Double JPEG Artifacts Removalcited by 3
  5. 2024CLIP-SR: Collaborative Linguistic and Image Processing for Super-Resolutioncited by 3
  6. 2025SRSR: Enhancing Semantic Accuracy in Real-World Image Super-Resolution with Spatially Re-Focused Text-Conditioningcited by 3
  7. 2025Semantic-Guided Cross-Sensor Super Resolution of Remote Sensing Images: A Gated Dual Conditioning Flow Matching Modelcited by 3
  8. 2026TPGDiff: Hierarchical Triple-Prior Guided Diffusion for Image Restorationcited by 3
  9. 2026FiDeSR: High-Fidelity and Detail-Preserving One-Step Diffusion Super-Resolutioncited by 3
  10. 2025Enhanced Semantic Extraction and Guidance for UGC Image Super Resolutioncited by 2
  11. 2025Adapting Large VLMs with Iterative and Manual Instructions for Generative Low-light Enhancementcited by 2
  12. 2025RAGSR: Regional Attention Guided Diffusion for Image Super-Resolutioncited by 2
  13. 2025DP2O-SR: Direct Perceptual Preference Optimization for Real-World Image Super-Resolutioncited by 2
  14. 2025Enhancing Diffusion-based Restoration Models via Difficulty-Adaptive Reinforcement Learning with IQA Rewardcited by 2
  15. 2026VOSR: A Vision-Only Generative Model for Image Super-Resolutioncited by 2
  16. 2024Zoomed In, Diffused Out: Towards Local Degradation-Aware Multi-Diffusion for Extreme Image Super-Resolutioncited by 1
  17. 2025Accelerating Diffusion-based Super-Resolution with Dynamic Time-Spatial Samplingcited by 1
  18. 2025Semantic and Temporal Integration in Latent Diffusion Space for High-Fidelity Video Super-Resolutioncited by 1
  19. 2025OmniScaleSR: Unleashing Scale-Controlled Diffusion Prior for Faithful and Realistic Arbitrary-Scale Image Super-Resolutioncited by 1
  20. 2026LucidNFT: LR-Anchored Multi-Reward Preference Optimization for Flow-Based Real-World Super-Resolutioncited by 1
  21. 2026DreamSR: Towards Ultra-High-Resolution Image Super-Resolution via a Receptive-Field Enhanced Diffusion Transformercited by 1
  22. 2025Score2Instruct: Scaling Up Video Quality-Centric Instructions via Automated Dimension Scoringcited by 0
  23. 2025Boosting Fidelity for Pre-Trained-Diffusion-Based Low-Light Image Enhancement via Condition Refinementcited by 0
  24. 2026QUSR: Quality-Aware and Uncertainty-Guided Image Super-Resolution Diffusion Modelcited by 0
  25. 2026Revisiting the Perception-Distortion Trade-off with Spatial-Semantic Guided Super-Resolutioncited by 0
  26. 2026Joint Degradation-Aware Arbitrary-Scale Super-Resolution for Variable-Rate Extreme Image Compressioncited by 0
  27. 2026Simon-SR: Spatially Adaptive Modulation and Visual Prompt Adaptation for Text-Reinforced Super-Resolutioncited by 0
  28. 2026LPM: Industrial-Scale Generative Video Restorationcited by 0
  29. 2026When Latents Forget Pixels: Restoring Fidelity in Diffusion Transformer Super-Resolutioncited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2024Taming Diffusion Prior for Image Super-Resolution with Domain Shift SDEs
  2. 2026Disentangled Textual Priors for Diffusion-based Image Super-Resolution
  3. 2024Improving Consistency in Diffusion Models for Image Super-Resolution
  4. 2026Decoupling Cross-Modality Manifold Discrepancy: Leveraging Visible Diffusion Priors for Infrared Super-Resolution
  5. 2024Degradation-Guided One-Step Image Super-Resolution with Diffusion Priors
  6. 2024ControlSR: Taming Diffusion Models for Consistent Real-World Image Super Resolution
  7. 2025Semantic-Guided Diffusion Model for Single-Step Image Super-Resolution
  8. 2024DaLPSR: Leverage Degradation-Aligned Language Prompt for Real-World Image Super-Resolution