# Rethinking Data Augmentation for Image Super-resolution: A Comprehensive Analysis and a New Strategy

arXiv: https://arxiv.org/abs/2004.00448 · HTML: https://mlsanity.com/paper/2004.00448/

## Key facts

- Cited by 66 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 1 Apr 2020.

## Built on by (most cited first, 30 of 66)

- [Data-centric Artificial Intelligence: A Survey](https://mlsanity.com/paper/2303.10158/) (2023)
- [Multi-Modal 3D Object Detection in Autonomous Driving: a Survey](https://mlsanity.com/paper/2106.12735/) (2021)
- [Global-Local Path Networks for Monocular Depth Estimation with Vertical CutDepth](https://mlsanity.com/paper/2201.07436/) (2022)
- [Avoiding Overfitting: A Survey on Regularization Methods for Convolutional Neural Networks](https://mlsanity.com/paper/2201.03299/) (2022)
- [Reflash Dropout in Image Super-Resolution](https://mlsanity.com/paper/2112.12089/) (2021)
- [A survey of synthetic data augmentation methods in computer vision](https://mlsanity.com/paper/2403.10075/) (2024)
- [A Survey of Mix-based Data Augmentation: Taxonomy, Methods, Applications, and Explainability](https://mlsanity.com/paper/2212.10888/) (2022)
- [IFQA: Interpretable Face Quality Assessment](https://mlsanity.com/paper/2211.07077/) (2022)
- [Image Data Augmentation Approaches: A Comprehensive Survey and Future directions](https://mlsanity.com/paper/2301.02830/) (2023)
- [Seeing What Matters: Generalizable AI-generated Video Detection with Forensic-Oriented Augmentation](https://mlsanity.com/paper/2506.16802/) (2025)
- [Learning Representational Invariances for Data-Efficient Action Recognition](https://mlsanity.com/paper/2103.16565/) (2021)
- [You Only Cut Once: Boosting Data Augmentation with a Single Cut](https://mlsanity.com/paper/2201.12078/) (2022)
- [A Two-Stage Attentive Network for Single Image Super-Resolution](https://mlsanity.com/paper/2104.10488/) (2021)
- [AIM 2022 Challenge on Super-Resolution of Compressed Image and Video: Dataset, Methods and Results](https://mlsanity.com/paper/2208.11184/) (2022)
- [A Survey on Mixup Augmentations and Beyond](https://mlsanity.com/paper/2409.05202/) (2024)
- [CutDepth:Edge-aware Data Augmentation in Depth Estimation](https://mlsanity.com/paper/2107.07684/) (2021)
- [D2HNet: Joint Denoising and Deblurring with Hierarchical Network for Robust Night Image Restoration](https://mlsanity.com/paper/2207.03294/) (2022)
- [MuS2: A Real-World Benchmark for Sentinel-2 Multi-Image Super-Resolution](https://mlsanity.com/paper/2210.02745/) (2022)
- [NTIRE 2022 Challenge on Stereo Image Super-Resolution: Methods and Results](https://mlsanity.com/paper/2204.09197/) (2022)
- [OccluMix: Towards De-Occlusion Virtual Try-on by Semantically-Guided Mixup](https://mlsanity.com/paper/2301.00965/) (2023)
- [Exploring the Application of Large-scale Pre-trained Models on Adverse Weather Removal](https://mlsanity.com/paper/2306.09008/) (2023)
- [Exploring Temporally Dynamic Data Augmentation for Video Recognition](https://mlsanity.com/paper/2206.15015/) (2022)
- [ESTformer: Transformer utilising spatiotemporal dependencies for electroencephalogram super-resolution](https://mlsanity.com/paper/2312.10052/) (2023)
- [Data Acquisition and Preparation for Dual-reference Deep Learning of Image Super-Resolution](https://mlsanity.com/paper/2108.02348/) (2021)
- [ObjectMix: Data Augmentation by Copy-Pasting Objects in Videos for Action Recognition](https://mlsanity.com/paper/2204.00239/) (2022)
- [An Efficient Framework for Few-shot Skeleton-based Temporal Action Segmentation](https://mlsanity.com/paper/2207.09925/) (2022)
- [NTIRE 2023 Challenge on Light Field Image Super-Resolution: Dataset, Methods and Results](https://mlsanity.com/paper/2304.10415/) (2023)
- [Learning Multiple Probabilistic Degradation Generators for Unsupervised Real World Image Super Resolution](https://mlsanity.com/paper/2201.10747/) (2022)
- [Unfolding Once is Enough: A Deployment-Friendly Transformer Unit for Super-Resolution](https://mlsanity.com/paper/2308.02794/) (2023)
- [Restoring Spatially-Heterogeneous Distortions using Mixture of Experts Network](https://mlsanity.com/paper/2009.14563/) (2020)

## Abstract (arXiv)

Data augmentation is an effective way to improve the performance of deep networks. Unfortunately, current methods are mostly developed for high-level vision tasks (e.g., classification) and few are studied for low-level vision tasks (e.g., image restoration). In this paper, we provide a comprehensive analysis of the existing augmentation methods applied to the super-resolution task. We find that the methods discarding or manipulating the pixels or features too much hamper the image restoration, where the spatial relationship is very important. Based on our analyses, we propose CutBlur that cuts a low-resolution patch and pastes it to the corresponding high-resolution image region and vice versa. The key intuition of CutBlur is to enable a model to learn not only "how" but also "where" to super-resolve an image. By doing so, the model can understand "how much", instead of blindly learning to apply super-resolution to every given pixel. Our method consistently and significantly improves the performance across various scenarios, especially when the model size is big and the data is collected under real-world environments. We also show that our method improves other low-level vision tasks, such as denoising and compression artifact removal.