# Learning to Resize Images for Computer Vision Tasks

arXiv: https://arxiv.org/abs/2103.09950 · HTML: https://mlsanity.com/paper/2103.09950/

## Key facts

- Cited by 45 papers in the map (data through 14 Oct 2026).
- Builds on 28 of its 47 references that are in the map.
- First posted on arXiv 17 Mar 2021.

## Builds on (most cited first, 28 of 28)

- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [ImageNet Large Scale Visual Recognition Challenge](https://mlsanity.com/paper/1409.0575/) (2014)
- [Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shift](https://mlsanity.com/paper/1502.03167/) (2015)
- [Densely Connected Convolutional Networks](https://mlsanity.com/paper/1608.06993/) (2016)
- [EfficientNet: Rethinking Model Scaling for Convolutional Neural Networks](https://mlsanity.com/paper/1905.11946/) (2019)
- [Rethinking the Inception Architecture for Computer Vision](https://mlsanity.com/paper/1512.00567/) (2015)
- [MobileNetV2: Inverted Residuals and Linear Bottlenecks](https://mlsanity.com/paper/1801.04381/) (2018)
- [Progressive Growing of GANs for Improved Quality, Stability, and Variation](https://mlsanity.com/paper/1710.10196/) (2017)
- [Photo-Realistic Single Image Super-Resolution Using a Generative Adversarial Network](https://mlsanity.com/paper/1609.04802/) (2016)
- [Enhanced Deep Residual Networks for Single Image Super-Resolution](https://mlsanity.com/paper/1707.02921/) (2017)
- [ESRGAN: Enhanced Super-Resolution Generative Adversarial Networks](https://mlsanity.com/paper/1809.00219/) (2018)
- [Toward Real-World Single Image Super-Resolution: A New Benchmark and A New Model](https://mlsanity.com/paper/1904.00523/) (2019)
- [NIMA: Neural Image Assessment](https://mlsanity.com/paper/1709.05424/) (2017)
- [Fixing the train-test resolution discrepancy](https://mlsanity.com/paper/1906.06423/) (2019)
- [Task-Driven Super Resolution: Object Detection in Low-resolution Images](https://mlsanity.com/paper/1803.11316/) (2018)
- [When Image Denoising Meets High-Level Vision Tasks: A Deep Learning Approach](https://mlsanity.com/paper/1706.04284/) (2017)
- [Super-Identity Convolutional Neural Network for Face Hallucination](https://mlsanity.com/paper/1811.02328/) (2018)
- [Disentangling Features in 3D Face Shapes for Joint Face Reconstruction and Recognition](https://mlsanity.com/paper/1803.11366/) (2018)
- [End-to-End United Video Dehazing and Detection](https://mlsanity.com/paper/1709.03919/) (2017)
- [The Rate-Distortion-Accuracy Tradeoff: JPEG Case Study](https://mlsanity.com/paper/2008.00605/) (2020)
- [Dual Directed Capsule Network for Very Low Resolution Image Recognition](https://mlsanity.com/paper/1908.10027/) (2019)
- [Classification Driven Dynamic Image Enhancement](https://mlsanity.com/paper/1710.07558/) (2017)
- [Bridging the Gap Between Computational Photography and Visual Recognition](https://mlsanity.com/paper/1901.09482/) (2019)
- [Better Compression with Deep Pre-Editing](https://mlsanity.com/paper/2002.00113/) (2020)
- [The Unreasonable Effectiveness of Texture Transfer for Single Image Super-resolution](https://mlsanity.com/paper/1808.00043/) (2018)
- [Creating High Resolution Images with a Latent Adversarial Generator](https://mlsanity.com/paper/2003.02365/) (2020)
- [Enhancing the Performance of Convolutional Neural Networks on Quality Degraded Datasets](https://mlsanity.com/paper/1710.06805/) (2017)
- [ThumbNet: One Thumbnail Image Contains All You Need for Recognition](https://mlsanity.com/paper/1904.05034/) (2019)

## Built on by (most cited first, 30 of 45)

- [MaxViT: Multi-Axis Vision Transformer](https://mlsanity.com/paper/2204.01697/) (2022)
- [Test-Time Adaptation to Distribution Shift by Confidence Maximization and Input Transformation](https://mlsanity.com/paper/2106.14999/) (2021)
- [Bridging the Domain Gap for Multi-Agent Perception](https://mlsanity.com/paper/2210.08451/) (2022)
- [Learning strides in convolutional neural networks](https://mlsanity.com/paper/2202.01653/) (2022)
- [Learning to Downsample for Segmentation of Ultra-High Resolution Images](https://mlsanity.com/paper/2109.11071/) (2021)
- [Constructive Distortion: Improving MLLMs with Attention-Guided Image Warping](https://mlsanity.com/paper/2510.09741/) (2025)
- [Distilling Knowledge from Object Classification to Aesthetics Assessment](https://mlsanity.com/paper/2206.00809/) (2022)
- [Class-level Structural Relation Modelling and Smoothing for Visual Representation Learning](https://mlsanity.com/paper/2308.04142/) (2023)
- [A Recent Survey of Heterogeneous Transfer Learning](https://mlsanity.com/paper/2310.08459/) (2023)
- [Joint Learning of Blind Super-Resolution and Crack Segmentation for Realistic Degraded Images](https://mlsanity.com/paper/2302.12491/) (2023)
- [Vision Transformer Equipped with Neural Resizer on Facial Expression Recognition Task](https://mlsanity.com/paper/2204.02181/) (2022)
- [Out of Distribution Detection via Neural Network Anchoring](https://mlsanity.com/paper/2207.04125/) (2022)
- [Exploring Resolution and Degradation Clues as Self-supervised Signal for Low Quality Object Detection](https://mlsanity.com/paper/2208.03062/) (2022)
- [Differentiable Sensor Layouts for End-to-End Learning of Task-Specific Camera Parameters](https://mlsanity.com/paper/2304.14736/) (2023)
- [MSPE: Multi-Scale Patch Embedding Prompts Vision Transformers to Any Resolution](https://mlsanity.com/paper/2405.18240/) (2024)
- [Repaint: Improving the Generalization of Down-Stream Visual Tasks by Generating Multiple Instances of Training Examples](https://mlsanity.com/paper/2110.10366/) (2021)
- [Patch-wise Contrastive Style Learning for Instagram Filter Removal](https://mlsanity.com/paper/2204.07486/) (2022)
- [Content-Adaptive Downsampling in Convolutional Neural Networks](https://mlsanity.com/paper/2305.09504/) (2023)
- [Charm: The Missing Piece in ViT fine-tuning for Image Aesthetic Assessment](https://mlsanity.com/paper/2504.02522/) (2025)
- [ResiDualGAN: Resize-Residual DualGAN for Cross-Domain Remote Sensing Images Semantic Segmentation](https://mlsanity.com/paper/2201.11523/) (2022)
- [Dynamic Test-Time Augmentation via Differentiable Functions](https://mlsanity.com/paper/2212.04681/) (2022)
- [MULLER: Multilayer Laplacian Resizer for Vision](https://mlsanity.com/paper/2304.02859/) (2023)
- [Real-World Atmospheric Turbulence Correction via Domain Adaptation](https://mlsanity.com/paper/2402.07371/) (2024)
- [UltraMNIST Classification: A Benchmark to Train CNNs for Very Large Images](https://mlsanity.com/paper/2206.12681/) (2022)
- [Simulating analogue film damage to analyse and improve artefact restoration on high-resolution scans](https://mlsanity.com/paper/2302.10004/) (2023)
- [AnimateZoo: Zero-shot Video Generation of Cross-Species Animation via Subject Alignment](https://mlsanity.com/paper/2404.04946/) (2024)
- [Learning Spatio-Temporal Downsampling for Effective Video Upscaling](https://mlsanity.com/paper/2203.08140/) (2022)
- [Test-time Adaptation Meets Image Enhancement: Improving Accuracy via Uncertainty-aware Logit Switching](https://mlsanity.com/paper/2403.17423/) (2024)
- [Sketch Down the FLOPs: Towards Efficient Networks for Human Sketch](https://mlsanity.com/paper/2505.23763/) (2025)
- [Kernel Inversed Pyramidal Resizing Network for Efficient Pavement Distress Recognition](https://mlsanity.com/paper/2212.01790/) (2022)

## Abstract (arXiv)

For all the ways convolutional neural nets have revolutionized computer vision in recent years, one important aspect has received surprisingly little attention: the effect of image size on the accuracy of tasks being trained for. Typically, to be efficient, the input images are resized to a relatively small spatial resolution (e.g. 224x224), and both training and inference are carried out at this resolution. The actual mechanism for this re-scaling has been an afterthought: Namely, off-the-shelf image resizers such as bilinear and bicubic are commonly used in most machine learning software frameworks. But do these resizers limit the on task performance of the trained networks? The answer is yes. Indeed, we show that the typical linear resizer can be replaced with learned resizers that can substantially improve performance. Importantly, while the classical resizers typically result in better perceptual quality of the downscaled images, our proposed learned resizers do not necessarily give better visual quality, but instead improve task performance. Our learned image resizer is jointly trained with a baseline vision model. This learned CNN-based resizer creates machine friendly visual manipulations that lead to a consistent improvement of the end task metric over the baseline model. Specifically, here we focus on the classification task with the ImageNet dataset, and experiment with four different models to learn resizers adapted to each model. Moreover, we show that the proposed resizer can also be useful for fine-tuning the classification baselines for other vision tasks. To this end, we experiment with three different baselines to develop image quality assessment (IQA) models on the AVA dataset.