# Reference-based Image Super-Resolution with Deformable Attention Transformer

arXiv: https://arxiv.org/abs/2207.11938 · HTML: https://mlsanity.com/paper/2207.11938/

## Key facts

- Cited by 51 papers in the map (data through 14 Oct 2026).
- Builds on 40 of its 61 references that are in the map.
- First posted on arXiv 25 Jul 2022.

## Builds on (most cited first, 30 of 40)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [Very Deep Convolutional Networks for Large-Scale Image Recognition](https://mlsanity.com/paper/1409.1556/) (2014)
- [Swin Transformer: Hierarchical Vision Transformer using Shifted Windows](https://mlsanity.com/paper/2103.14030/) (2021)
- [The Unreasonable Effectiveness of Deep Features as a Perceptual Metric](https://mlsanity.com/paper/1801.03924/) (2018)
- [Photo-Realistic Single Image Super-Resolution Using a Generative Adversarial Network](https://mlsanity.com/paper/1609.04802/) (2016)
- [SwinIR: Image Restoration Using Swin Transformer](https://mlsanity.com/paper/2108.10257/) (2021)
- [Enhanced Deep Residual Networks for Single Image Super-Resolution](https://mlsanity.com/paper/1707.02921/) (2017)
- [Deformable Convolutional Networks](https://mlsanity.com/paper/1703.06211/) (2017)
- [Image Super-Resolution Using Very Deep Residual Channel Attention Networks](https://mlsanity.com/paper/1807.02758/) (2018)
- [ESRGAN: Enhanced Super-Resolution Generative Adversarial Networks](https://mlsanity.com/paper/1809.00219/) (2018)
- [Image Super-Resolution Using Deep Convolutional Networks](https://mlsanity.com/paper/1501.00092/) (2014)
- [Swin-Unet: Unet-like Pure Transformer for Medical Image Segmentation](https://mlsanity.com/paper/2105.05537/) (2021)
- [Deep Image Prior](https://mlsanity.com/paper/1711.10925/) (2017)
- [Video Swin Transformer](https://mlsanity.com/paper/2106.13230/) (2021)
- [Real-ESRGAN: Training Real-World Blind Super-Resolution with Pure Synthetic Data](https://mlsanity.com/paper/2107.10833/) (2021)
- [Deformable ConvNets v2: More Deformable, Better Results](https://mlsanity.com/paper/1811.11168/) (2018)
- [Designing a Practical Degradation Model for Deep Blind Image Super-Resolution](https://mlsanity.com/paper/2103.14006/) (2021)
- [FFDNet: Toward a Fast and Flexible Solution for CNN based Image Denoising](https://mlsanity.com/paper/1710.04026/) (2017)
- [EDVR: Video Restoration with Enhanced Deformable Convolutional Networks](https://mlsanity.com/paper/1905.02716/) (2019)
- [Learning Texture Transformer Network for Image Super-Resolution](https://mlsanity.com/paper/2006.04139/) (2020)
- [BasicVSR++: Improving Video Super-Resolution with Enhanced Propagation and Alignment](https://mlsanity.com/paper/2104.13371/) (2021)
- [Sketch-based Manga Retrieval using Manga109 Dataset](https://mlsanity.com/paper/1510.04389/) (2015)
- [EnhanceNet: Single Image Super-Resolution Through Automated Texture Synthesis](https://mlsanity.com/paper/1612.07919/) (2016)
- [Feedback Network for Image Super-Resolution](https://mlsanity.com/paper/1903.09814/) (2019)
- [RankSRGAN: Generative Adversarial Networks with Ranker for Image Super-Resolution](https://mlsanity.com/paper/1908.06382/) (2019)
- [Image Super-Resolution by Neural Texture Transfer](https://mlsanity.com/paper/1903.00834/) (2019)
- [CrossNet: An End-to-end Reference-based Super Resolution Network using Cross-scale Warping](https://mlsanity.com/paper/1807.10547/) (2018)
- [Closed-loop Matters: Dual Regression Networks for Single Image Super-Resolution](https://mlsanity.com/paper/2003.07018/) (2020)
- [NTIRE 2020 Challenge on Real-World Image Super-Resolution: Methods and Results](https://mlsanity.com/paper/2005.01996/) (2020)
- [ClassSR: A General Framework to Accelerate Super-Resolution Networks by Data Characteristic](https://mlsanity.com/paper/2103.04039/) (2021)

## Built on by (most cited first, 30 of 51)

- [CiaoSR: Continuous Implicit Attention-in-Attention Network for Arbitrary-Scale Image Super-Resolution](https://mlsanity.com/paper/2212.04362/) (2022)
- [CoSeR: Bridging Image and Language for Cognitive Super-Resolution](https://mlsanity.com/paper/2311.16512/) (2023)
- [Improving Image Restoration through Removing Degradations in Textual Representations](https://mlsanity.com/paper/2312.17334/) (2023)
- [Building Bridges across Spatial and Temporal Resolutions: Reference-Based Super-Resolution via Change Priors and Conditional Diffusion Model](https://mlsanity.com/paper/2403.17460/) (2024)
- [Unaligned RGB Guided Hyperspectral Image Super-Resolution with Spatial-Spectral Concordance](https://mlsanity.com/paper/2505.02109/) (2025)
- [Deep Equilibrium Diffusion Restoration with Parallel Sampling](https://mlsanity.com/paper/2311.11600/) (2023)
- [ReFIR: Grounding Large Restoration Models with Retrieval Augmentation](https://mlsanity.com/paper/2410.05601/) (2024)
- [LMR: A Large-Scale Multi-Reference Dataset for Reference-based Super-Resolution](https://mlsanity.com/paper/2303.04970/) (2023)
- [WM-MoE: Weather-aware Multi-scale Mixture-of-Experts for Blind Adverse Weather Removal](https://mlsanity.com/paper/2303.13739/) (2023)
- [From NeRFLiX to NeRFLiX++: A General NeRF-Agnostic Restorer Paradigm](https://mlsanity.com/paper/2306.06388/) (2023)
- [Trust but Verify: Adaptive Conditioning for Reference-Based Diffusion Super-Resolution via Implicit Reference Correlation Modeling](https://mlsanity.com/paper/2602.01864/) (2026)
- [KeDuSR: Real-World Dual-Lens Super-Resolution via Kernel-Free Matching](https://mlsanity.com/paper/2312.17050/) (2023)
- [MaRINeR: Enhancing Novel Views by Matching Rendered Images with Nearby References](https://mlsanity.com/paper/2407.13745/) (2024)
- [DRM-IR: Task-Adaptive Deep Unfolding Network for All-In-One Image Restoration](https://mlsanity.com/paper/2307.07688/) (2023)
- [Bridging the Domain Gap: A Simple Domain Matching Method for Reference-based Image Super-Resolution in Remote Sensing](https://mlsanity.com/paper/2401.15944/) (2024)
- [Self-Supervised Learning for Real-World Super-Resolution from Dual and Multiple Zoomed Observations](https://mlsanity.com/paper/2405.02171/) (2024)
- [HMANet: Hybrid Multi-Axis Aggregation Network for Image Super-Resolution](https://mlsanity.com/paper/2405.05001/) (2024)
- [InstantIR: Blind Image Restoration with Instant Generative Reference](https://mlsanity.com/paper/2410.06551/) (2024)
- [SRSplat: Feed-Forward Super-Resolution Gaussian Splatting from Sparse Multi-View Images](https://mlsanity.com/paper/2511.12040/) (2025)
- [A Feature Reuse Framework with Texture-adaptive Aggregation for Reference-based Super-Resolution](https://mlsanity.com/paper/2306.01500/) (2023)
- [DARTS: Double Attention Reference-based Transformer for Super-resolution](https://mlsanity.com/paper/2307.08837/) (2023)
- [Joint Flow And Feature Refinement Using Attention For Video Restoration](https://mlsanity.com/paper/2505.16434/) (2025)
- [Reference-based OCT Angiogram Super-resolution with Learnable Texture Generation](https://mlsanity.com/paper/2305.05835/) (2023)
- [DeeDSR: Towards Real-World Image Super-Resolution via Degradation-Aware Stable Diffusion](https://mlsanity.com/paper/2404.00661/) (2024)
- [DCAE-SR: Design of a Denoising Convolutional Autoencoder for reconstructing Electrocardiograms signals at Super Resolution](https://mlsanity.com/paper/2404.15307/) (2024)
- [Detail-Enhancing Framework for Reference-Based Image Super-Resolution](https://mlsanity.com/paper/2405.00431/) (2024)
- [HiTSR: A Hierarchical Transformer for Reference-based Super-Resolution](https://mlsanity.com/paper/2408.16959/) (2024)
- [Ultra-High-Definition Reference-Based Landmark Image Super-Resolution with Generative Diffusion Prior](https://mlsanity.com/paper/2508.10779/) (2025)
- [Equivariant Sampling for Improving Diffusion Model-based Image Restoration](https://mlsanity.com/paper/2511.09965/) (2025)
- [INSIGHT: An Interpretable Neural Vision-Language Framework for Reasoning of Generative Artifacts](https://mlsanity.com/paper/2511.22351/) (2025)

## Abstract (arXiv)

Reference-based image super-resolution (RefSR) aims to exploit auxiliary reference (Ref) images to super-resolve low-resolution (LR) images. Recently, RefSR has been attracting great attention as it provides an alternative way to surpass single image SR. However, addressing the RefSR problem has two critical challenges: (i) It is difficult to match the correspondence between LR and Ref images when they are significantly different; (ii) How to transfer the relevant texture from Ref images to compensate the details for LR images is very challenging. To address these issues of RefSR, this paper proposes a deformable attention Transformer, namely DATSR, with multiple scales, each of which consists of a texture feature encoder (TFE) module, a reference-based deformable attention (RDA) module and a residual feature aggregation (RFA) module. Specifically, TFE first extracts image transformation (e.g., brightness) insensitive features for LR and Ref images, RDA then can exploit multiple relevant textures to compensate more information for LR features, and RFA lastly aggregates LR features and relevant textures to get a more visually pleasant result. Extensive experiments demonstrate that our DATSR achieves state-of-the-art performance on benchmark datasets quantitatively and qualitatively.