# Image Super-Resolution with Text Prompt Diffusion

arXiv: https://arxiv.org/abs/2311.14282 · HTML: https://mlsanity.com/paper/2311.14282/

## Key facts

- Cited by 23 papers in the map (data through 14 Oct 2026).
- Builds on 69 of its 81 references that are in the map.
- First posted on arXiv 24 Nov 2023.

## Builds on (most cited first, 30 of 69)

- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [Learning Transferable Visual Models From Natural Language Supervision](https://mlsanity.com/paper/2103.00020/) (2021)
- [Denoising Diffusion Probabilistic Models](https://mlsanity.com/paper/2006.11239/) (2020)
- [PyTorch: An Imperative Style, High-Performance Deep Learning Library](https://mlsanity.com/paper/1912.01703/) (2019)
- [U-Net: Convolutional Networks for Biomedical Image Segmentation](https://mlsanity.com/paper/1505.04597/) (2015)
- [High-Resolution Image Synthesis with Latent Diffusion Models](https://mlsanity.com/paper/2112.10752/) (2021)
- [GPT-4 Technical Report](https://mlsanity.com/paper/2303.08774/) (2023)
- [Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformer](https://mlsanity.com/paper/1910.10683/) (2019)
- [The Unreasonable Effectiveness of Deep Features as a Perceptual Metric](https://mlsanity.com/paper/1801.03924/) (2018)
- [Denoising Diffusion Implicit Models](https://mlsanity.com/paper/2010.02502/) (2020)
- [Visual Instruction Tuning](https://mlsanity.com/paper/2304.08485/) (2023)
- [Hierarchical Text-Conditional Image Generation with CLIP Latents](https://mlsanity.com/paper/2204.06125/) (2022)
- [Photorealistic Text-to-Image Diffusion Models with Deep Language Understanding](https://mlsanity.com/paper/2205.11487/) (2022)
- [A Style-Based Generator Architecture for Generative Adversarial Networks](https://mlsanity.com/paper/1812.04948/) (2018)
- [Adding Conditional Control to Text-to-Image Diffusion Models](https://mlsanity.com/paper/2302.05543/) (2023)
- [BLIP: Bootstrapping Language-Image Pre-training for Unified Vision-Language Understanding and Generation](https://mlsanity.com/paper/2201.12086/) (2022)
- [Zero-Shot Text-to-Image Generation](https://mlsanity.com/paper/2102.12092/) (2021)
- [Improved Denoising Diffusion Probabilistic Models](https://mlsanity.com/paper/2102.09672/) (2021)
- [InstructPix2Pix: Learning to Follow Image Editing Instructions](https://mlsanity.com/paper/2211.09800/) (2022)
- [Prompt-to-Prompt Image Editing with Cross Attention Control](https://mlsanity.com/paper/2208.01626/) (2022)
- [SwinIR: Image Restoration Using Swin Transformer](https://mlsanity.com/paper/2108.10257/) (2021)
- [Image Super-Resolution via Iterative Refinement](https://mlsanity.com/paper/2104.07636/) (2021)
- [RePaint: Inpainting using Denoising Diffusion Probabilistic Models](https://mlsanity.com/paper/2201.09865/) (2022)
- [DiffWave: A Versatile Diffusion Model for Audio Synthesis](https://mlsanity.com/paper/2009.09761/) (2020)
- [Image Super-Resolution Using Very Deep Residual Channel Attention Networks](https://mlsanity.com/paper/1807.02758/) (2018)
- [Imagic: Text-Based Real Image Editing with Diffusion Models](https://mlsanity.com/paper/2210.09276/) (2022)
- [Diffusion-LM Improves Controllable Text Generation](https://mlsanity.com/paper/2205.14217/) (2022)
- [StyleCLIP: Text-Driven Manipulation of StyleGAN Imagery](https://mlsanity.com/paper/2103.17249/) (2021)

## Built on by (most cited first, 23 of 23)

- [Degradation-Guided One-Step Image Super-Resolution with Diffusion Priors](https://mlsanity.com/paper/2409.17058/) (2024)
- [XPSR: Cross-modal Priors for Diffusion-based Image Super-Resolution](https://mlsanity.com/paper/2403.05049/) (2024)
- [SPIRE: Semantic Prompt-Driven Image Restoration](https://mlsanity.com/paper/2312.11595/) (2023)
- [Teaching Tailored to Talent: Adverse Weather Restoration via Prompt Pool and Depth-Anything Constraint](https://mlsanity.com/paper/2409.15739/) (2024)
- [OFTSR: One-Step Flow for Image Super-Resolution with Tunable Fidelity-Realism Trade-offs](https://mlsanity.com/paper/2412.09465/) (2024)
- [DiffRetouch: Using Diffusion to Retouch on the Shoulder of Experts](https://mlsanity.com/paper/2407.03757/) (2024)
- [Overcoming False Illusions in Real-World Face Restoration with Multi-Modal Guided Diffusion Model](https://mlsanity.com/paper/2410.04161/) (2024)
- [Instruct-IPT: All-in-One Image Processing Transformer via Weight Modulation](https://mlsanity.com/paper/2407.00676/) (2024)
- [Dog-IQA: Standard-guided Zero-shot MLLM for Mix-grained Image Quality Assessment](https://mlsanity.com/paper/2410.02505/) (2024)
- [Acquire and then Adapt: Squeezing out Text-to-Image Model for Image Restoration](https://mlsanity.com/paper/2504.15159/) (2025)
- [HazeCLIP: Towards Language Guided Real-World Image Dehazing](https://mlsanity.com/paper/2407.13719/) (2024)
- [Human Body Restoration with One-Step Diffusion Model and A New Benchmark](https://mlsanity.com/paper/2502.01411/) (2025)
- [RealSR-R1: Reinforcement Learning for Real-World Image Super-Resolution with Vision-Language Chain-of-Thought](https://mlsanity.com/paper/2506.16796/) (2025)
- [Tell Me What You See: Text-Guided Real-World Image Denoising](https://mlsanity.com/paper/2312.10191/) (2023)
- [DeeDSR: Towards Real-World Image Super-Resolution via Degradation-Aware Stable Diffusion](https://mlsanity.com/paper/2404.00661/) (2024)
- [DIffSteISR: Harnessing Diffusion Prior for Superior Real-world Stereo Image Super-Resolution](https://mlsanity.com/paper/2408.07516/) (2024)
- [HonestFace: Towards Honest Face Restoration with One-Step Diffusion Model](https://mlsanity.com/paper/2505.18469/) (2025)
- [Referring Flexible Image Restoration](https://mlsanity.com/paper/2404.10342/) (2024)
- [DR-BFR: Degradation Representation with Diffusion Models for Blind Face Restoration](https://mlsanity.com/paper/2411.10508/) (2024)
- [Pathology Image Restoration via Mixture of Prompts](https://mlsanity.com/paper/2503.12399/) (2025)
- [TextOVSR: Text-Guided Real-World Opera Video Super-Resolution](https://mlsanity.com/paper/2603.15153/) (2026)
- [Face2Scene: Using Facial Degradation as an Oracle for Diffusion-Based Scene Restoration](https://mlsanity.com/paper/2603.16570/) (2026)
- [Adaptive Context Matters: Towards Provable Multi-Modality Guidance for Super-Resolution](https://mlsanity.com/paper/2605.10470/) (2026)

## Abstract (arXiv)

Image super-resolution (SR) methods typically model degradation to improve reconstruction accuracy in complex and unknown degradation scenarios. However, extracting degradation information from low-resolution images is challenging, which limits the model performance. To boost image SR performance, one feasible approach is to introduce additional priors. Inspired by advancements in multi-modal methods and text prompt image processing, we introduce text prompts to image SR to provide degradation priors. Specifically, we first design a text-image generation pipeline to integrate text into the SR dataset through the text degradation representation and degradation model. By adopting a discrete design, the text representation is flexible and user-friendly. Meanwhile, we propose the PromptSR to realize the text prompt SR. The PromptSR leverages the latest multi-modal large language model (MLLM) to generate prompts from low-resolution images. It also utilizes the pre-trained language model (e.g., T5 or CLIP) to enhance text comprehension. We train the PromptSR on the text-image dataset. Extensive experiments indicate that introducing text prompts into SR, yields impressive results on both synthetic and real-world images. Code: https://github.com/zhengchen1999/PromptSR.