Rethinking Super-Resolution as Text-Guided Details Generation
Not on the map. Why?
Abstract (arXiv)
Deep neural networks have greatly promoted the performance of single image super-resolution (SISR). Conventional methods still resort to restoring the single high-resolution (HR) solution only based on the input of image modality. However, the image-level information is insufficient to predict adequate details and photo-realistic visual quality facing large upscaling factors (x8, x16). In this paper, we propose a new perspective that regards the SISR as a semantic image detail enhancement problem to generate semantically reasonable HR image that are faithful to the ground truth. To enhance the semantic accuracy and the visual quality of the reconstructed image, we explore the multi-modal fusion learning in SISR by proposing a Text-Guided Super-Resolution (TGSR) framework, which can effectively utilize the information from the text and image modalities. Different from existing methods, the proposed TGSR could generate HR image details that match the text descriptions through a coarse-to-fine process. Extensive experiments and ablation studies demonstrate the effect of the TGSR, which exploits the text reference to recover realistic images.
Key facts. Cited by 3 papers in the map (data through 7 Oct 2026). Builds on 29 of its 38 references that are in the map. First posted on arXiv 14 Jul 2022.
Builds on
29 of 38 references.
References · 29
- 2014Microsoft COCO: Common Objects in Contextcited by 17,498
- 2022Generative Adversarial Networkscited by 5,603
- 2015Rethinking the Inception Architecture for Computer Visioncited by 5,451
- 2014Deep Learning Face Attributes in the Wildcited by 3,679
- 2016Photo-Realistic Single Image Super-Resolution Using a Generative Adversarial Networkcited by 1,898
- 2017Enhanced Deep Residual Networks for Single Image Super-Resolutioncited by 1,543
- 2018ESRGAN: Enhanced Super-Resolution Generative Adversarial Networkscited by 1,250
- 2014Image Super-Resolution Using Deep Convolutional Networkscited by 1,218
- 2015Accurate Image Super-Resolution Using Very Deep Convolutional Networkscited by 902
- 2016Generative Adversarial Text to Image Synthesiscited by 735
- 2017AttnGAN: Fine-Grained Text to Image Generation with Attentional Generative Adversarial Networkscited by 664
- 2016StackGAN: Text to Photo-realistic Image Synthesis with Stacked Generative Adversarial Networkscited by 575
17 more
- 2018Recovering Realistic Texture in Image Super-resolution by Deep Spatial Feature Transformcited by 369
- 2020PULSE: Self-Supervised Photo Upsampling via Latent Space Exploration of Generative Modelscited by 358
- 2021Cross-Modal Contrastive Learning for Text-to-Image Generationcited by 259
- 2020PIPAL: a Large-Scale Image Quality Assessment Dataset for Perceptual Image Restorationcited by 177
- 2017FSRNet: End-to-End Learning Face Super-Resolution with Facial Priorscited by 170
- 2019ManiGAN: Text-Guided Image Manipulationcited by 156
- 2019Object-driven Text-to-Image Synthesis via Adversarial Trainingcited by 117
- 2020Structure-Preserving Super Resolution with Gradient Guidancecited by 116
- 2020Closed-loop Matters: Dual Regression Networks for Single Image Super-Resolutioncited by 90
- 2018Deep Semantic Face Deblurringcited by 80
- 2020Learning Meta Face Recognition in Unseen Domainscited by 61
- 2017Super-FAN: Integrated facial landmark localization and super-resolution of real-world low resolution faces in arbitrary poses with GANscited by 55
- 2017Language-Based Image Editing with Recurrent Attentive Modelscited by 53
- 2020Deep Face Super-Resolution with Iterative Collaboration between Attentive Recovery and Landmark Estimationcited by 53
- 2019Explorable Super Resolutioncited by 44
- 2019Tag2Pix: Line Art Colorization Using Text Tag With SECat and Changing Losscited by 35
- 2020DeepSEE: Deep Disentangled Semantic Explorative Extreme Super-Resolutioncited by 17
Built on by
Citing papers per year: 2023: 1; 2024: 2.
3 papers.