# Image Super-Resolution by Neural Texture Transfer

arXiv: https://arxiv.org/abs/1903.00834 · HTML: https://mlsanity.com/paper/1903.00834/

## Key facts

- Cited by 120 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 3 Mar 2019.

## Built on by (most cited first, 30 of 120)

- [Exploiting Diffusion Prior for Real-World Image Super-Resolution](https://mlsanity.com/paper/2305.07015/) (2023)
- [Real-World Blind Super-Resolution via Feature Matching with Implicit High-Resolution Priors](https://mlsanity.com/paper/2202.13142/) (2022)
- [NeRF-SR: High-Quality Neural Radiance Fields using Supersampling](https://mlsanity.com/paper/2112.01759/) (2021)
- [MASA-SR: Matching Acceleration and Spatial Adaptation for Reference-Based Image Super-Resolution](https://mlsanity.com/paper/2106.02299/) (2021)
- [Robust Reference-based Super-Resolution via C2-Matching](https://mlsanity.com/paper/2106.01863/) (2021)
- [CoSeR: Bridging Image and Language for Cognitive Super-Resolution](https://mlsanity.com/paper/2311.16512/) (2023)
- [Reference-based Image Super-Resolution with Deformable Attention Transformer](https://mlsanity.com/paper/2207.11938/) (2022)
- [Implicit Transformer Network for Screen Content Image Continuous Super-Resolution](https://mlsanity.com/paper/2112.06174/) (2021)
- [Language-Driven Image Style Transfer](https://mlsanity.com/paper/2106.00178/) (2021)
- [HQG-Net: Unpaired Medical Image Enhancement with High-Quality Guidance](https://mlsanity.com/paper/2307.07829/) (2023)
- [Dual-Camera Super-Resolution with Aligned Attention Modules](https://mlsanity.com/paper/2109.01349/) (2021)
- [LocalTrans: A Multiscale Local Transformer Network for Cross-Resolution Homography Estimation](https://mlsanity.com/paper/2106.04067/) (2021)
- [DynaST: Dynamic Sparse Transformer for Exemplar-Guided Image Generation](https://mlsanity.com/paper/2207.06124/) (2022)
- [Attention-Guided Progressive Neural Texture Fusion for High Dynamic Range Image Restoration](https://mlsanity.com/paper/2107.06211/) (2021)
- [Spatial-Temporal Super-Resolution of Satellite Imagery via Conditional Pixel Synthesis](https://mlsanity.com/paper/2106.11485/) (2021)
- [Self-Supervised Learning for Real-World Super-Resolution from Dual Zoomed Observations](https://mlsanity.com/paper/2203.01325/) (2022)
- [Reference-based Video Super-Resolution Using Multi-Camera Video Triplets](https://mlsanity.com/paper/2203.14537/) (2022)
- [A Systematic Survey of Deep Learning-based Single-Image Super-Resolution](https://mlsanity.com/paper/2109.14335/) (2021)
- [Coarse-to-Fine Embedded PatchMatch and Multi-Scale Dynamic Aggregation for Reference-based Super-Resolution](https://mlsanity.com/paper/2201.04358/) (2022)
- [GLEAN: Generative Latent Bank for Image Super-Resolution and Beyond](https://mlsanity.com/paper/2207.14812/) (2022)
- [Variational AutoEncoder for Reference based Image Super-Resolution](https://mlsanity.com/paper/2106.04090/) (2021)
- [Continuous Remote Sensing Image Super-Resolution based on Context Interaction in Implicit Function Space](https://mlsanity.com/paper/2302.08046/) (2023)
- [Memory-Augmented Non-Local Attention for Video Super-Resolution](https://mlsanity.com/paper/2108.11048/) (2021)
- [Precise Facial Landmark Detection by Reference Heatmap Transformer](https://mlsanity.com/paper/2303.07840/) (2023)
- [ReFIR: Grounding Large Restoration Models with Retrieval Augmentation](https://mlsanity.com/paper/2410.05601/) (2024)
- [PetsGAN: Rethinking Priors for Single Image Generation](https://mlsanity.com/paper/2203.01488/) (2022)
- [Attention-based Multi-Reference Learning for Image Super-Resolution](https://mlsanity.com/paper/2108.13697/) (2021)
- [RRSR:Reciprocal Reference-based Image Super-Resolution with Progressive Feature Alignment and Selection](https://mlsanity.com/paper/2211.04203/) (2022)
- [LMR: A Large-Scale Multi-Reference Dataset for Reference-based Super-Resolution](https://mlsanity.com/paper/2303.04970/) (2023)
- [Efficient Space-time Video Super Resolution using Low-Resolution Flow and Mask Upsampling](https://mlsanity.com/paper/2104.05778/) (2021)

## Abstract (arXiv)

Due to the significant information loss in low-resolution (LR) images, it has become extremely challenging to further advance the state-of-the-art of single image super-resolution (SISR). Reference-based super-resolution (RefSR), on the other hand, has proven to be promising in recovering high-resolution (HR) details when a reference (Ref) image with similar content as that of the LR input is given. However, the quality of RefSR can degrade severely when Ref is less similar. This paper aims to unleash the potential of RefSR by leveraging more texture details from Ref images with stronger robustness even when irrelevant Ref images are provided. Inspired by the recent work on image stylization, we formulate the RefSR problem as neural texture transfer. We design an end-to-end deep model which enriches HR details by adaptively transferring the texture from Ref images according to their textural similarity. Instead of matching content in the raw pixel space as done by previous methods, our key contribution is a multi-level matching conducted in the neural space. This matching scheme facilitates multi-scale neural transfer that allows the model to benefit more from those semantically related Ref patches, and gracefully degrade to SISR performance on the least relevant Ref inputs. We build a benchmark dataset for the general research of RefSR, which contains Ref images paired with LR inputs with varying levels of similarity. Both quantitative and qualitative evaluations demonstrate the superiority of our method over state-of-the-art.