# Variational AutoEncoder for Reference based Image Super-Resolution

arXiv: https://arxiv.org/abs/2106.04090 · HTML: https://mlsanity.com/paper/2106.04090/

## Key facts

- Cited by 18 papers in the map (data through 14 Oct 2026).
- Builds on 26 of its 44 references that are in the map.
- First posted on arXiv 8 Jun 2021.

## Builds on (most cited first, 26 of 26)

- [Very Deep Convolutional Networks for Large-Scale Image Recognition](https://mlsanity.com/paper/1409.1556/) (2014)
- [The Unreasonable Effectiveness of Deep Features as a Perceptual Metric](https://mlsanity.com/paper/1801.03924/) (2018)
- [Perceptual Losses for Real-Time Style Transfer and Super-Resolution](https://mlsanity.com/paper/1603.08155/) (2016)
- [Photo-Realistic Single Image Super-Resolution Using a Generative Adversarial Network](https://mlsanity.com/paper/1609.04802/) (2016)
- [Enhanced Deep Residual Networks for Single Image Super-Resolution](https://mlsanity.com/paper/1707.02921/) (2017)
- [Image Super-Resolution Using Very Deep Residual Channel Attention Networks](https://mlsanity.com/paper/1807.02758/) (2018)
- [ESRGAN: Enhanced Super-Resolution Generative Adversarial Networks](https://mlsanity.com/paper/1809.00219/) (2018)
- [Image Super-Resolution Using Deep Convolutional Networks](https://mlsanity.com/paper/1501.00092/) (2014)
- [High-Resolution Image Synthesis and Semantic Manipulation with Conditional GANs](https://mlsanity.com/paper/1711.11585/) (2017)
- [Accurate Image Super-Resolution Using Very Deep Convolutional Networks](https://mlsanity.com/paper/1511.04587/) (2015)
- [Residual Dense Network for Image Super-Resolution](https://mlsanity.com/paper/1802.08797/) (2018)
- [The Perception-Distortion Tradeoff](https://mlsanity.com/paper/1711.06077/) (2017)
- [Deep Laplacian Pyramid Networks for Fast and Accurate Super-Resolution](https://mlsanity.com/paper/1704.03915/) (2017)
- [Single Image Super-Resolution via a Holistic Attention Network](https://mlsanity.com/paper/2008.08767/) (2020)
- [The relativistic discriminator: a key element missing from standard GAN](https://mlsanity.com/paper/1807.00734/) (2018)
- [SRFlow: Learning the Super-Resolution Space with Normalizing Flow](https://mlsanity.com/paper/2006.14200/) (2020)
- [Arbitrary Style Transfer with Style-Attentional Networks](https://mlsanity.com/paper/1812.02342/) (2018)
- [Image Super-Resolution by Neural Texture Transfer](https://mlsanity.com/paper/1903.00834/) (2019)
- [CrossNet: An End-to-end Reference-based Super Resolution Network using Cross-scale Warping](https://mlsanity.com/paper/1807.10547/) (2018)
- [Explorable Super Resolution](https://mlsanity.com/paper/1912.01839/) (2019)
- [Latent Constraints: Learning to Generate Conditionally from Unconditional Generative Models](https://mlsanity.com/paper/1711.05772/) (2017)
- [ESRGAN+ : Further Improving Enhanced Super-Resolution Generative Adversarial Network](https://mlsanity.com/paper/2001.08073/) (2020)
- [DeepSEE: Deep Disentangled Semantic Explorative Extreme Super-Resolution](https://mlsanity.com/paper/2004.04433/) (2020)
- [Image Super-Resolution via Attention based Back Projection Networks](https://mlsanity.com/paper/1910.04476/) (2019)
- [Unsupervised Real Image Super-Resolution via Generative Variational AutoEncoder](https://mlsanity.com/paper/2004.12811/) (2020)
- [Hierarchical Back Projection Network for Image Super-Resolution](https://mlsanity.com/paper/1906.06874/) (2019)

## Built on by (most cited first, 18 of 18)

- [Advancements in Generative AI: A Comprehensive Review of GANs, GPT, Autoencoders, Diffusion Model, and Transformers](https://mlsanity.com/paper/2311.10242/) (2023)
- [Separable Multi-Concept Erasure from Diffusion Models](https://mlsanity.com/paper/2402.05947/) (2024)
- [Reasons for the Superiority of Stochastic Estimators over Deterministic Ones: Robustness, Consistency and Perceptual Quality](https://mlsanity.com/paper/2211.08944/) (2022)
- [How do Variational Autoencoders Learn? Insights from Representational Similarity](https://mlsanity.com/paper/2205.08399/) (2022)
- [Diffusion Prior Interpolation for Flexibility Real-World Face Super-Resolution](https://mlsanity.com/paper/2412.16552/) (2024)
- [Stochasticity-aware No-Reference Point Cloud Quality Assessment](https://mlsanity.com/paper/2401.08926/) (2024)
- [Deep Spatio-Temporal Neural Network for Air Quality Reanalysis](https://mlsanity.com/paper/2502.11941/) (2025)
- [How good are variational autoencoders at transfer learning?](https://mlsanity.com/paper/2304.10767/) (2023)
- [Laplacian Pyramid-like Autoencoder](https://mlsanity.com/paper/2208.12484/) (2022)
- [Leveraging Vision-Language Models to Select Trustworthy Super-Resolution Samples Generated by Diffusion Models](https://mlsanity.com/paper/2506.20832/) (2025)
- [A Comprehensive Review of Diffusion Models in Smart Agriculture: Progress, Applications, and Challenges](https://mlsanity.com/paper/2507.18376/) (2025)
- [Super-resolution Reconstruction of Single Image for Latent features](https://mlsanity.com/paper/2211.12845/) (2022)
- [Trustworthy SR: Resolving Ambiguity in Image Super-resolution via Diffusion Models and Human Feedback](https://mlsanity.com/paper/2402.07597/) (2024)
- [β-Multivariational Autoencoder for Entangled Representation Learning in Video Frames](https://mlsanity.com/paper/2211.12627/) (2022)
- [On the Adversarial Robustness of Generative Autoencoders in the Latent Space](https://mlsanity.com/paper/2307.02202/) (2023)
- [Domain Transfer in Latent Space (DTLS) Wins on Image Super-Resolution -- a Non-Denoising Model](https://mlsanity.com/paper/2311.02358/) (2023)
- [VisioBlend: Sketch and Stroke-Guided Denoising Diffusion Probabilistic Model for Realistic Image Generation](https://mlsanity.com/paper/2407.05209/) (2024)
- [How to Re-enable PDE Loss for Physical Systems Modeling Under Partial Observation](https://mlsanity.com/paper/2412.09116/) (2024)

## Abstract (arXiv)

In this paper, we propose a novel reference based image super-resolution approach via Variational AutoEncoder (RefVAE). Existing state-of-the-art methods mainly focus on single image super-resolution which cannot perform well on large upsampling factors, e.g., 8×. We propose a reference based image super-resolution, for which any arbitrary image can act as a reference for super-resolution. Even using random map or low-resolution image itself, the proposed RefVAE can transfer the knowledge from the reference to the super-resolved images. Depending upon different references, the proposed method can generate different versions of super-resolved images from a hidden super-resolution space. Besides using different datasets for some standard evaluations with PSNR and SSIM, we also took part in the NTIRE2021 SR Space challenge and have provided results of the randomness evaluation of our approach. Compared to other state-of-the-art methods, our approach achieves higher diverse scores.