mlsanityData to 7 Oct 2026

TextIR: A Simple Framework for Text-based Editable Image Restoration

arXiv 2302.14736 ↗First posted 28 Feb 2023cs.CVcited by 17

Not on the map. Why?

Abstract (arXiv)

Most existing image restoration methods use neural networks to learn strong image-level priors from huge data to estimate the lost information. However, these works still struggle in cases when images have severe information deficits. Introducing external priors or using reference images to provide information also have limitations in the application domain. In contrast, text input is more readily available and provides information with higher flexibility. In this work, we design an effective framework that allows the user to control the restoration process of degraded images with text descriptions. We use the text-image feature compatibility of the CLIP to alleviate the difficulty of fusing text and image features. Our framework can be used for various image restoration tasks, including image inpainting, image super-resolution, and image colorization. Extensive experiments demonstrate the effectiveness of our method.

Key facts. Cited by 17 papers in the map (data through 7 Oct 2026). Builds on 65 of its 91 references that are in the map. First posted on arXiv 28 Feb 2023.

Builds on

65 of 91 references.

References · 65

  1. 2017Attention Is All You Needcited by 59,932
  2. 2014Adam: A Method for Stochastic Optimizationcited by 36,960
  3. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,456
  4. 2020Denoising Diffusion Probabilistic Modelscited by 20,851
  5. 2021High-Resolution Image Synthesis with Latent Diffusion Modelscited by 18,694
  6. 2018The Unreasonable Effectiveness of Deep Features as a Perceptual Metriccited by 11,029
  7. 2021Diffusion Models Beat GANs on Image Synthesiscited by 8,185
  8. 2022Hierarchical Text-Conditional Image Generation with CLIP Latentscited by 6,539
  9. 2022Photorealistic Text-to-Image Diffusion Models with Deep Language Understandingcited by 6,322
  10. 2023BLIP-2: Bootstrapping Language-Image Pre-training with Frozen Image Encoders and Large Language Modelscited by 6,069
  11. 2018A Style-Based Generator Architecture for Generative Adversarial Networkscited by 5,986
  12. 2017Neural Discrete Representation Learningcited by 4,650
38 more
  1. 2022Flamingo: a Visual Language Model for Few-Shot Learningcited by 4,337
  2. 2021Zero-Shot Text-to-Image Generationcited by 4,276
  3. 2021Improved Denoising Diffusion Probabilistic Modelscited by 3,736
  4. 2021GLIDE: Towards Photorealistic Image Generation and Editing with Text-Guided Diffusion Modelscited by 3,595
  5. 2019Analyzing and Improving the Image Quality of StyleGANcited by 3,580
  6. 2020Taming Transformers for High-Resolution Image Synthesiscited by 3,229
  7. 2016Perceptual Losses for Real-Time Style Transfer and Super-Resolutioncited by 2,678
  8. 2018Large Scale GAN Training for High Fidelity Natural Image Synthesiscited by 2,609
  9. 2019ViLBERT: Pretraining Task-Agnostic Visiolinguistic Representations for Vision-and-Language Taskscited by 2,219
  10. 2021SwinIR: Image Restoration Using Swin Transformercited by 1,869
  11. 2021Restormer: Efficient Transformer for High-Resolution Image Restorationcited by 1,618
  12. 2019LXMERT: Learning Cross-Modality Encoder Representations from Transformerscited by 1,548
  13. 2019Generating Diverse High-Fidelity Images with VQ-VAE-2cited by 1,376
  14. 2014Image Super-Resolution Using Deep Convolutional Networkscited by 1,218
  15. 2020Improved Techniques for Training Score-Based Generative Modelscited by 1,036
  16. 2021StyleCLIP: Text-Driven Manipulation of StyleGAN Imagerycited by 990
  17. 2019VL-BERT: Pre-training of Generic Visual-Linguistic Representationscited by 954
  18. 2021Blended Diffusion for Text-driven Editing of Natural Imagescited by 924
  19. 2021Uformer: A General U-Shaped Transformer for Image Restorationcited by 864
  20. 2016COCO-Stuff: Thing and Stuff Classes in Contextcited by 858
  21. 2021Florence: A New Foundation Model for Computer Visioncited by 802
  22. 2016Generative Adversarial Text to Image Synthesiscited by 735
  23. 2017AttnGAN: Fine-Grained Text to Image Generation with Attentional Generative Adversarial Networkscited by 664
  24. 2016StackGAN: Text to Photo-realistic Image Synthesis with Stacked Generative Adversarial Networkscited by 575
  25. 2018Image Inpainting for Irregular Holes Using Partial Convolutionscited by 505
  26. 2021CLIP-NeRF: Text-and-Image Driven Manipulation of Neural Radiance Fieldscited by 369
  27. 2021Towards Real-World Blind Face Restoration with Generative Facial Priorcited by 347
  28. 2021Rethinking Coarse-to-Fine Approach in Single Image Deblurringcited by 324
  29. 2020Learning Enriched Features for Real Image Restoration and Enhancementcited by 311
  30. 2022CogView2: Faster and Better Text-to-Image Generation via Hierarchical Transformerscited by 305
  31. 2022VQGAN-CLIP: Open Domain Image Generation and Editing with Natural Language Guidancecited by 285
  32. 2019Residual Non-local Attention Networks for Image Restorationcited by 236
  33. 2022Towards Robust Blind Face Restoration with Codebook Lookup Transformercited by 230
  34. 2017StackGAN++: Realistic Image Synthesis with Stacked Generative Adversarial Networkscited by 229
  35. 2021More Control for Free! Image Synthesis with Semantic Diffusion Guidancecited by 211
  36. 2018Recurrent Squeeze-and-Excitation Context Aggregation Net for Single Image Derainingcited by 205
  37. 2021CLIPstyler: Image Style Transfer with a Single Text Conditioncited by 203
  38. 2021GAN Prior Embedded Network for Blind Face Restoration in the Wildcited by 197

50 most cited of 65 All 65 (JSON)

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2023: 8; 2024: 3; 2025: 2; 2026: 4.

17 papers.

Sort:

By area:

  1. 2024InstructIR: High-Quality Image Restoration Following Human Instructionscited by 139
  2. 2023DreamDiffusion: Generating High-Quality Images from Brain EEG Signalscited by 56
  3. 2023Improving Image Restoration through Removing Degradations in Textual Representationscited by 36
  4. 2023EventBind: Learning a Unified Representation to Bind Them All for Event-based Open-world Understandingcited by 33
  5. 2023Regularization by Texts for Latent Diffusion Inverse Solverscited by 31
  6. 2023Textual Prompt Guided Image Restorationcited by 27
  7. 2023SPIRE: Semantic Prompt-Driven Image Restorationcited by 24
  8. 2024AWRaCLe: All-Weather Image Restoration using Visual In-Context Learningcited by 18
  9. 2023From Posterior Sampling to Meaningful Diversity in Image Restorationcited by 12
  10. 2023Tell Me What You See: Text-Guided Real-World Image Denoisingcited by 2
  11. 2024Strong and Controllable Blind Image Decompositioncited by 2
  12. 2025VL-UR: Vision-Language-guided Universal Restoration of Images Degraded by Adverse Weather Conditionscited by 2
5 more
  1. 2025DreamPose3D: Hallucinative Diffusion with Prompt Learning for 3D Human Pose Estimationcited by 2
  2. 2026Vision-Language Model Purified Semi-Supervised Semantic Segmentation for Remote Sensing Imagescited by 1
  3. 2026Look, Compare and Draw: Differential Query Transformer for Automatic Oil Paintingcited by 1
  4. 2026From Competition to Coopetition: Coopetitive Training-Free Image Editing Based on Text Guidancecited by 1
  5. 2026MDTD-ArtIR: Benchmarking Image Editing and Restoration Models for Art Image Restoration under Texture-Overlay Degradationscited by 0