# Coarse-to-Fine Embedded PatchMatch and Multi-Scale Dynamic Aggregation for Reference-based Super-Resolution

arXiv: https://arxiv.org/abs/2201.04358 · HTML: https://mlsanity.com/paper/2201.04358/

## Key facts

- Cited by 19 papers in the map (data through 14 Oct 2026).
- Builds on 25 of its 34 references that are in the map.
- First posted on arXiv 12 Jan 2022.

## Builds on (most cited first, 25 of 25)

- [Very Deep Convolutional Networks for Large-Scale Image Recognition](https://mlsanity.com/paper/1409.1556/) (2014)
- [Distilling the Knowledge in a Neural Network](https://mlsanity.com/paper/1503.02531/) (2015)
- [Momentum Contrast for Unsupervised Visual Representation Learning](https://mlsanity.com/paper/1911.05722/) (2019)
- [Perceptual Losses for Real-Time Style Transfer and Super-Resolution](https://mlsanity.com/paper/1603.08155/) (2016)
- [Visualizing and Understanding Convolutional Networks](https://mlsanity.com/paper/1311.2901/) (2013)
- [Improved Training of Wasserstein GANs](https://mlsanity.com/paper/1704.00028/) (2017)
- [Photo-Realistic Single Image Super-Resolution Using a Generative Adversarial Network](https://mlsanity.com/paper/1609.04802/) (2016)
- [Enhanced Deep Residual Networks for Single Image Super-Resolution](https://mlsanity.com/paper/1707.02921/) (2017)
- [Deformable Convolutional Networks](https://mlsanity.com/paper/1703.06211/) (2017)
- [Image Super-Resolution Using Very Deep Residual Channel Attention Networks](https://mlsanity.com/paper/1807.02758/) (2018)
- [ESRGAN: Enhanced Super-Resolution Generative Adversarial Networks](https://mlsanity.com/paper/1809.00219/) (2018)
- [Image Super-Resolution Using Deep Convolutional Networks](https://mlsanity.com/paper/1501.00092/) (2014)
- [Accurate Image Super-Resolution Using Very Deep Convolutional Networks](https://mlsanity.com/paper/1511.04587/) (2015)
- [Residual Dense Network for Image Super-Resolution](https://mlsanity.com/paper/1802.08797/) (2018)
- [Learning Texture Transformer Network for Image Super-Resolution](https://mlsanity.com/paper/2006.04139/) (2020)
- [Sketch-based Manga Retrieval using Manga109 Dataset](https://mlsanity.com/paper/1510.04389/) (2015)
- [PatchmatchNet: Learned Multi-View Patchmatch Stereo](https://mlsanity.com/paper/2012.01411/) (2020)
- [EnhanceNet: Single Image Super-Resolution Through Automated Texture Synthesis](https://mlsanity.com/paper/1612.07919/) (2016)
- [RankSRGAN: Generative Adversarial Networks with Ranker for Image Super-Resolution](https://mlsanity.com/paper/1908.06382/) (2019)
- [Image Super-Resolution by Neural Texture Transfer](https://mlsanity.com/paper/1903.00834/) (2019)
- [DeepPruner: Learning Efficient Stereo Matching via Differentiable PatchMatch](https://mlsanity.com/paper/1909.05845/) (2019)
- [CrossNet: An End-to-end Reference-based Super Resolution Network using Cross-scale Warping](https://mlsanity.com/paper/1807.10547/) (2018)
- [MASA-SR: Matching Acceleration and Spatial Adaptation for Reference-Based Image Super-Resolution](https://mlsanity.com/paper/2106.02299/) (2021)
- [Robust Reference-based Super-Resolution via C2-Matching](https://mlsanity.com/paper/2106.01863/) (2021)
- [Learning with Privileged Information for Efficient Image Super-Resolution](https://mlsanity.com/paper/2007.07524/) (2020)

## Built on by (most cited first, 19 of 19)

- [Any-resolution Training for High-resolution Image Synthesis](https://mlsanity.com/paper/2204.07156/) (2022)
- [CoSeR: Bridging Image and Language for Cognitive Super-Resolution](https://mlsanity.com/paper/2311.16512/) (2023)
- [Knowledge Distillation based Degradation Estimation for Blind Super-Resolution](https://mlsanity.com/paper/2211.16928/) (2022)
- [Basic Binary Convolution Unit for Binarized Image Restoration Network](https://mlsanity.com/paper/2210.00405/) (2022)
- [Building Bridges across Spatial and Temporal Resolutions: Reference-Based Super-Resolution via Change Priors and Conditional Diffusion Model](https://mlsanity.com/paper/2403.17460/) (2024)
- [LMR: A Large-Scale Multi-Reference Dataset for Reference-based Super-Resolution](https://mlsanity.com/paper/2303.04970/) (2023)
- [A Dive into SAM Prior in Image Restoration](https://mlsanity.com/paper/2305.13620/) (2023)
- [Efficient Hybrid Zoom using Camera Fusion on Mobile Phones](https://mlsanity.com/paper/2401.01461/) (2024)
- [InstantIR: Blind Image Restoration with Instant Generative Reference](https://mlsanity.com/paper/2410.06551/) (2024)
- [Multi-Reference Image Super-Resolution: A Posterior Fusion Approach](https://mlsanity.com/paper/2212.09988/) (2022)
- [Detail-Enhancing Framework for Reference-Based Image Super-Resolution](https://mlsanity.com/paper/2405.00431/) (2024)
- [HSTR-Net: Reference Based Video Super-resolution with Dual Cameras](https://mlsanity.com/paper/2310.12092/) (2023)
- [The Devil is in the Details: Boosting Guided Depth Super-Resolution via Rethinking Cross-Modal Alignment and Aggregation](https://mlsanity.com/paper/2401.08123/) (2024)
- [Sewer Image Super-Resolution with Depth Priors and Its Lightweight Network](https://mlsanity.com/paper/2407.19271/) (2024)
- [GMT: Enhancing Generalizable Neural Rendering via Geometry-Driven Multi-Reference Texture Transfer](https://mlsanity.com/paper/2410.00672/) (2024)
- [DM3Net: Dual-Camera Super-Resolution via Domain Modulation and Multi-scale Matching](https://mlsanity.com/paper/2506.06993/) (2025)
- [Segmentation-Aware Latent Diffusion for Satellite Image Super-Resolution: Enabling Smallholder Farm Boundary Delineation](https://mlsanity.com/paper/2511.14481/) (2025)
- [Learning to Balance: Decoupled Siamese Diffusion Transformer for Reference-Based Remote Sensing Image Super-Resolution](https://mlsanity.com/paper/2605.17980/) (2026)
- [DuSRFlow: Dual-Lens Super-Resolution via Parallax-Aware Flow Estimation and Complementary Reference Warping](https://mlsanity.com/paper/2610.14630/) (2026)

## Abstract (arXiv)

Reference-based super-resolution (RefSR) has made significant progress in producing realistic textures using an external reference (Ref) image. However, existing RefSR methods obtain high-quality correspondence matchings consuming quadratic computation resources with respect to the input size, limiting its application. Moreover, these approaches usually suffer from scale misalignments between the low-resolution (LR) image and Ref image. In this paper, we propose an Accelerated Multi-Scale Aggregation network (AMSA) for Reference-based Super-Resolution, including Coarse-to-Fine Embedded PatchMatch (CFE-PatchMatch) and Multi-Scale Dynamic Aggregation (MSDA) module. To improve matching efficiency, we design a novel Embedded PatchMacth scheme with random samples propagation, which involves end-to-end training with asymptotic linear computational cost to the input size. To further reduce computational cost and speed up convergence, we apply the coarse-to-fine strategy on Embedded PatchMacth constituting CFE-PatchMatch. To fully leverage reference information across multiple scales and enhance robustness to scale misalignment, we develop the MSDA module consisting of Dynamic Aggregation and Multi-Scale Aggregation. The Dynamic Aggregation corrects minor scale misalignment by dynamically aggregating features, and the Multi-Scale Aggregation brings robustness to large scale misalignment by fusing multi-scale information. Experimental results show that the proposed AMSA achieves superior performance over state-of-the-art approaches on both quantitative and qualitative evaluations.