mlsanityData to 14 Oct 2026

HoliSDiP: Image Super-Resolution via Holistic Semantics and Diffusion Prior

arXiv 2411.18662 ↗First posted 27 Nov 2024cs.CVcited by 8

On the map: Diffusion models → Inverse problems and image restoration

Abstract (arXiv)

Text-to-image diffusion models have emerged as powerful priors for real-world image super-resolution (Real-ISR). However, existing methods may produce unintended results due to noisy text prompts and their lack of spatial information. In this paper, we present HoliSDiP, a framework that leverages semantic segmentation to provide both precise textual and spatial guidance for diffusion-based Real-ISR. Our method employs semantic labels as concise text prompts while introducing dense semantic guidance through segmentation masks and our proposed Segmentation-CLIP Map. Extensive experiments demonstrate that HoliSDiP achieves significant improvement in image quality across various Real-ISR scenarios through reduced prompt noise and enhanced spatial control.

Key facts. Cited by 8 papers in the map (data through 14 Oct 2026). Placed in Inverse problems and image restoration, one of 14 directions in Diffusion models. Builds on 62 of its 70 references that are in the map. First posted on arXiv 27 Nov 2024.

Builds on

62 of 70 references.

Closest ancestors · 9

Same direction, newest first

  1. 2024One-Step Effective Diffusion Network for Real-World Image Super-Resolution12 Jun
  2. 2024XPSR: Cross-modal Priors for Diffusion-based Image Super-Resolution8 Mar
  3. 2023SeeSR: Towards Semantics-Aware Real-World Image Super-Resolution27 Nov
  4. 2023SinSR: Diffusion-Based Image Super-Resolution in a Single Step23 Nov
  5. 2023DiffBIR: Towards Blind Image Restoration with Generative Diffusion Prior29 Aug
  6. 2023Pixel-Aware Stable Diffusion for Realistic Image Super-resolution and Personalized Stylization28 Aug
  7. 2023ResShift: Efficient Diffusion Model for Image Super-resolution by Residual Shifting23 Jul
  8. 2023Exploiting Diffusion Prior for Real-World Image Super-Resolution11 May
  9. 2021SRDiff: Single Image Super-Resolution with Diffusion Probabilistic Models30 Apr

Other references · 53

  1. 2015Deep Residual Learning for Image Recognitioncited by 52,096
  2. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,744
  3. 2020Denoising Diffusion Probabilistic Modelscited by 21,076
  4. 2021High-Resolution Image Synthesis with Latent Diffusion Modelscited by 18,863
  5. 2021Swin Transformer: Hierarchical Vision Transformer using Shifted Windowscited by 12,139
  6. 2018The Unreasonable Effectiveness of Deep Features as a Perceptual Metriccited by 11,177
  7. 2023BLIP-2: Bootstrapping Language-Image Pre-training with Frozen Image Encoders and Large Language Modelscited by 6,137
  8. 2018A Style-Based Generator Architecture for Generative Adversarial Networkscited by 6,013
  9. 2023Adding Conditional Control to Text-to-Image Diffusion Modelscited by 5,503
  10. 2015You Only Look Once: Unified, Real-Time Object Detectioncited by 4,852
  11. 2023Improved Baselines with Visual Instruction Tuningcited by 4,418
  12. 2021Improved Denoising Diffusion Probabilistic Modelscited by 3,761
38 more
  1. 2021Masked-attention Mask Transformer for Universal Image Segmentationcited by 2,351
  2. 2016Photo-Realistic Single Image Super-Resolution Using a Generative Adversarial Networkcited by 1,908
  3. 2021SwinIR: Image Restoration Using Swin Transformercited by 1,887
  4. 2017Decoupled Weight Decay Regularizationcited by 1,831
  5. 2017Enhanced Deep Residual Networks for Single Image Super-Resolutioncited by 1,552
  6. 2021Image Super-Resolution via Iterative Refinementcited by 1,511
  7. 2019Semantic Image Synthesis with Spatially-Adaptive Normalizationcited by 1,299
  8. 2018ESRGAN: Enhanced Super-Resolution Generative Adversarial Networkscited by 1,254
  9. 2014Image Super-Resolution Using Deep Convolutional Networkscited by 1,224
  10. 2017Deep Image Priorcited by 1,098
  11. 2021MUSIQ: Multi-scale Image Quality Transformercited by 1,081
  12. 2022Exploring CLIP for Assessing the Look and Feel of Imagescited by 991
  13. 2021Real-ESRGAN: Training Real-World Blind Super-Resolution with Pure Synthetic Datacited by 970
  14. 2018Residual Dense Network for Image Super-Resolutioncited by 799
  15. 2017The Perception-Distortion Tradeoffcited by 587
  16. 2020Learning Continuous Image Representation with Local Implicit Image Functioncited by 552
  17. 2021Designing a Practical Degradation Model for Deep Blind Image Super-Resolutioncited by 529
  18. 2022MANIQA: Multi-dimension Attention Network for No-Reference Image Quality Assessmentcited by 522
  19. 2019Toward Real-World Single Image Super-Resolution: A New Benchmark and A New Modelcited by 403
  20. 2018Recovering Realistic Texture in Image Super-resolution by Deep Spatial Feature Transformcited by 371
  21. 2023Recognize Anything: A Strong Image Tagging Modelcited by 309
  22. 2017"Zero-Shot" Super-Resolution using Deep Internal Learningcited by 286
  23. 2024Scaling Up to Excellence: Practicing Model Scaling for Photo-Realistic Image Restoration In the Wildcited by 286
  24. 2020Component Divide-and-Conquer for Real-World Image Super-Resolutioncited by 246
  25. 2020SRFlow: Learning the Super-Resolution Space with Normalizing Flowcited by 209
  26. 2019Blind Super-Resolution With Iterative Kernel Correctioncited by 203
  27. 2019Meta-SR: A Magnification-Arbitrary Network for Super-Resolutioncited by 181
  28. 2019Blind Super-Resolution Kernel Estimation using an Internal-GANcited by 175
  29. 2020GLEAN: Generative Latent Bank for Large-Factor Image Super-Resolutioncited by 158
  30. 2019RankSRGAN: Generative Adversarial Networks with Ranker for Image Super-Resolutioncited by 145
  31. 2018Unsupervised Image Super-Resolution using Cycle-in-Cycle Generative Adversarial Networkscited by 133
  32. 2021Local Texture Estimator for Implicit Representation Functioncited by 127
  33. 2020Meta-Transfer Learning for Zero-Shot Super-Resolutioncited by 116
  34. 2019Frequency Separation for Real-World Super-Resolutioncited by 115
  35. 2022Real-World Blind Super-Resolution via Feature Matching with Implicit High-Resolution Priorscited by 100
  36. 2018To learn image super-resolution, use a GAN to learn how to do image degradation firstcited by 94
  37. 2021Hierarchical Conditional Flow: A Unified Framework for Image Super-Resolution and Image Rescalingcited by 73
  38. 2022CiaoSR: Continuous Implicit Attention-in-Attention Network for Arbitrary-Scale Image Super-Resolutioncited by 70

50 most cited of 53 All 53 (JSON)

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2025: 6; 2026: 2.

8 papers.

  1. 2025DiT4SR: Taming Diffusion Transformer for Real-World Image Super-Resolutioncited by 53
  2. 2025Semantic-Guided Diffusion Model for Single-Step Image Super-Resolutioncited by 3
  3. 2025Optimization-Free Universal Watermark Forgery with Regenerative Diffusion Modelscited by 3
  4. 2025SRSR: Enhancing Semantic Accuracy in Real-World Image Super-Resolution with Spatially Re-Focused Text-Conditioningcited by 3
  5. 2026Disentangled Textual Priors for Diffusion-based Image Super-Resolutioncited by 1
  6. 2026DreamSR: Towards Ultra-High-Resolution Image Super-Resolution via a Receptive-Field Enhanced Diffusion Transformercited by 1
  7. 2025MegaSR: Mining Customized Semantics and Expressive Guidance for Real-World Image Super-Resolutioncited by 0
  8. 2025Boosting Fidelity for Pre-Trained-Diffusion-Based Low-Light Image Enhancement via Condition Refinementcited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2024Semantic Segmentation Prior for Diffusion-Based Real-World Super-Resolution
  2. 2024Improving Consistency in Diffusion Models for Image Super-Resolution
  3. 2025Text-Aware Real-World Image Super-Resolution via Diffusion Model with Joint Segmentation Decoders
  4. 2023TextDiff: Mask-Guided Residual Diffusion Models for Scene Text Image Super-Resolution
  5. 2025Local-Global Context-Aware and Structure-Preserving Image Super-Resolution
  6. 2026Preserve More Details: Mitigating Content Drift in Real-World Image Super-Resolution
  7. 2024SAM-DiffSR: Structure-Modulated Diffusion Model for Image Super-Resolution
  8. 2024CasSR: Activating Image Power for Real-World Image Super-Resolution