# Data Acquisition and Preparation for Dual-reference Deep Learning of Image Super-Resolution

arXiv: https://arxiv.org/abs/2108.02348 · HTML: https://mlsanity.com/paper/2108.02348/

## Key facts

- Cited by 7 papers in the map (data through 14 Oct 2026).
- Builds on 29 of its 58 references that are in the map.
- First posted on arXiv 5 Aug 2021.

## Builds on (most cited first, 29 of 29)

- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale](https://mlsanity.com/paper/2010.11929/) (2020)
- [Perceptual Losses for Real-Time Style Transfer and Super-Resolution](https://mlsanity.com/paper/1603.08155/) (2016)
- [Photo-Realistic Single Image Super-Resolution Using a Generative Adversarial Network](https://mlsanity.com/paper/1609.04802/) (2016)
- [Enhanced Deep Residual Networks for Single Image Super-Resolution](https://mlsanity.com/paper/1707.02921/) (2017)
- [Image Super-Resolution Using Very Deep Residual Channel Attention Networks](https://mlsanity.com/paper/1807.02758/) (2018)
- [ESRGAN: Enhanced Super-Resolution Generative Adversarial Networks](https://mlsanity.com/paper/1809.00219/) (2018)
- [Image Super-Resolution Using Deep Convolutional Networks](https://mlsanity.com/paper/1501.00092/) (2014)
- [Real-Time Single Image and Video Super-Resolution Using an Efficient Sub-Pixel Convolutional Neural Network](https://mlsanity.com/paper/1609.05158/) (2016)
- [Deep Image Prior](https://mlsanity.com/paper/1711.10925/) (2017)
- [Accurate Image Super-Resolution Using Very Deep Convolutional Networks](https://mlsanity.com/paper/1511.04587/) (2015)
- [Residual Dense Network for Image Super-Resolution](https://mlsanity.com/paper/1802.08797/) (2018)
- [Toward Real-World Single Image Super-Resolution: A New Benchmark and A New Model](https://mlsanity.com/paper/1904.00523/) (2019)
- [Learning Texture Transformer Network for Image Super-Resolution](https://mlsanity.com/paper/2006.04139/) (2020)
- [Deeply-Recursive Convolutional Network for Image Super-Resolution](https://mlsanity.com/paper/1511.04491/) (2015)
- ["Zero-Shot" Super-Resolution using Deep Internal Learning](https://mlsanity.com/paper/1712.06087/) (2017)
- [EnhanceNet: Single Image Super-Resolution Through Automated Texture Synthesis](https://mlsanity.com/paper/1612.07919/) (2016)
- [Non-Local Recurrent Network for Image Restoration](https://mlsanity.com/paper/1806.02919/) (2018)
- [The 2018 PIRM Challenge on Perceptual Image Super-resolution](https://mlsanity.com/paper/1809.07517/) (2018)
- [Unsupervised Image Super-Resolution using Cycle-in-Cycle Generative Adversarial Networks](https://mlsanity.com/paper/1809.00437/) (2018)
- [Meta-Transfer Learning for Zero-Shot Super-Resolution](https://mlsanity.com/paper/2002.12213/) (2020)
- [Zoom To Learn, Learn To Zoom](https://mlsanity.com/paper/1905.05169/) (2019)
- [Frequency Separation for Real-World Super-Resolution](https://mlsanity.com/paper/1911.07850/) (2019)
- [To learn image super-resolution, use a GAN to learn how to do image degradation first](https://mlsanity.com/paper/1807.11458/) (2018)
- [Cross-Scale Internal Graph Neural Network for Image Super-Resolution](https://mlsanity.com/paper/2006.16673/) (2020)
- [Rethinking Data Augmentation for Image Super-resolution: A Comprehensive Analysis and a New Strategy](https://mlsanity.com/paper/2004.00448/) (2020)
- [Learning Parallax Attention for Stereo Image Super-Resolution](https://mlsanity.com/paper/1903.05784/) (2019)
- [Camera Lens Super-Resolution](https://mlsanity.com/paper/1904.03378/) (2019)
- [Towards Real Scene Super-Resolution with Raw Images](https://mlsanity.com/paper/1905.12156/) (2019)

## Built on by (most cited first, 7 of 7)

- [NTIRE 2024 Challenge on Image Super-Resolution (x4): Methods and Results](https://mlsanity.com/paper/2404.09790/) (2024)
- [NTIRE 2025 Challenge on Image Super-Resolution (x4): Methods and Results](https://mlsanity.com/paper/2504.14582/) (2025)
- [HIPA: Hierarchical Patch Transformer for Single Image Super Resolution](https://mlsanity.com/paper/2203.10247/) (2022)
- [Learning Degradation-Independent Representations for Camera ISP Pipelines](https://mlsanity.com/paper/2307.00761/) (2023)
- [Fast Point Cloud Geometry Compression with Context-based Residual Coding and INR-based Refinement](https://mlsanity.com/paper/2408.02966/) (2024)
- [Self-Supervised Face Image Restoration with a One-Shot Reference](https://mlsanity.com/paper/2203.03005/) (2022)
- [Revealing Microscopic Objects in Fluorescence Live Imaging by Video-to-video Translation Based on A Spatial-temporal Generative Adversarial Network](https://mlsanity.com/paper/2502.16342/) (2025)

## Abstract (arXiv)

The performance of deep learning based image super-resolution (SR) methods depend on how accurately the paired low and high resolution images for training characterize the sampling process of real cameras. Low and high resolution (LR~HR) image pairs synthesized by degradation models (e.g., bicubic downsampling) deviate from those in reality; thus the synthetically-trained DCNN SR models work disappointingly when being applied to real-world images. To address this issue, we propose a novel data acquisition process to shoot a large set of LR~HR image pairs using real cameras. The images are displayed on an ultra-high quality screen and captured at different resolutions. The resulting LR~HR image pairs can be aligned at very high sub-pixel precision by a novel spatial-frequency dual-domain registration method, and hence they provide more appropriate training data for the learning task of super-resolution. Moreover, the captured HR image and the original digital image offer dual references to strengthen supervised learning. Experimental results show that training a super-resolution DCNN by our LR~HR dataset achieves higher image quality than training it by other datasets in the literature. Moreover, the proposed screen-capturing data collection process can be automated; it can be carried out for any target camera with ease and low cost, offering a practical way of tailoring the training of a DCNN SR model separately to each of the given cameras.