# Learning When and Where to Zoom with Deep Reinforcement Learning

arXiv: https://arxiv.org/abs/2003.00425 · HTML: https://mlsanity.com/paper/2003.00425/

## Key facts

- Cited by 43 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 1 Mar 2020.

## Built on by (most cited first, 30 of 43)

- [AdaViT: Adaptive Vision Transformers for Efficient Image Recognition](https://mlsanity.com/paper/2111.15668/) (2021)
- [ImageNet-Hard: The Hardest Images Remaining from a Study of the Power of Zoom and Spatial Biases in Image Classification](https://mlsanity.com/paper/2304.05538/) (2023)
- [FOVEA: Foveated Image Magnification for Autonomous Navigation](https://mlsanity.com/paper/2108.12102/) (2021)
- [Dynamic Resolution Network](https://mlsanity.com/paper/2106.02898/) (2021)
- [Task-Oriented Image Transmission for Scene Classification in Unmanned Aerial Systems](https://mlsanity.com/paper/2112.10948/) (2021)
- [Resolution learning in deep convolutional networks using scale-space theory](https://mlsanity.com/paper/2106.03412/) (2021)
- [Hard-Attention for Scalable Image Classification](https://mlsanity.com/paper/2102.10212/) (2021)
- [Efficient Classification of Very Large Images with Tiny Objects](https://mlsanity.com/paper/2106.02694/) (2021)
- [IS-COUNT: Large-scale Object Counting from Satellite Images with Covariate-based Importance Sampling](https://mlsanity.com/paper/2112.09126/) (2021)
- [A Probabilistic Hard Attention Model For Sequentially Observed Scenes](https://mlsanity.com/paper/2111.07534/) (2021)
- [AdaZoom: Adaptive Zoom Network for Multi-Scale Object Detection in Large Scenes](https://mlsanity.com/paper/2106.10409/) (2021)
- [Consistency driven Sequential Transformers Attention Model for Partially Observable Scenes](https://mlsanity.com/paper/2204.00656/) (2022)
- [GOHSP: A Unified Framework of Graph and Optimization-based Heterogeneous Structured Pruning for Vision Transformer](https://mlsanity.com/paper/2301.05345/) (2023)
- [Efficient High-Resolution Deep Learning: A Survey](https://mlsanity.com/paper/2207.13050/) (2022)
- [A Visual Active Search Framework for Geospatial Exploration](https://mlsanity.com/paper/2211.15788/) (2022)
- [A Partially Supervised Reinforcement Learning Framework for Visual Active Search](https://mlsanity.com/paper/2310.09689/) (2023)
- [AdaGlimpse: Active Visual Exploration with Arbitrary Glimpse Position and Scale](https://mlsanity.com/paper/2404.03482/) (2024)
- [Transformer-based Context Condensation for Boosting Feature Pyramids in Object Detection](https://mlsanity.com/paper/2207.06603/) (2022)
- [GQHAN: A Grover-inspired Quantum Hard Attention Network](https://mlsanity.com/paper/2401.14089/) (2024)
- [LookWhere? Efficient Visual Recognition by Learning Where to Look and What to See from Self-Supervision](https://mlsanity.com/paper/2505.18051/) (2025)
- [Aerial View Localization with Reinforcement Learning: Towards Emulating Search-and-Rescue](https://mlsanity.com/paper/2209.03694/) (2022)
- [Using Distance Correlation for Efficient Bayesian Optimization](https://mlsanity.com/paper/2102.08993/) (2021)
- [RLCorrector: Reinforced Proofreading for Cell-level Microscopy Image Segmentation](https://mlsanity.com/paper/2106.05487/) (2021)
- [HCMS: Hierarchical and Conditional Modality Selection for Efficient Video Recognition](https://mlsanity.com/paper/2104.09760/) (2021)
- [Dynamic Low-Resolution Distillation for Cost-Efficient End-to-End Text Spotting](https://mlsanity.com/paper/2207.06694/) (2022)
- [SpectralZoom: Efficient Segmentation with an Adaptive Hyperspectral Camera](https://mlsanity.com/paper/2406.04287/) (2024)
- [Scale-Aware Recognition in Satellite Images under Resource Constraints](https://mlsanity.com/paper/2411.00210/) (2024)
- [Sketch Down the FLOPs: Towards Efficient Networks for Human Sketch](https://mlsanity.com/paper/2505.23763/) (2025)
- [SaccadeCam: Adaptive Visual Attention for Monocular Depth Sensing](https://mlsanity.com/paper/2103.12981/) (2021)
- [You Better Look Twice: a new perspective for designing accurate detectors with reduced computations](https://mlsanity.com/paper/2107.10050/) (2021)

## Abstract (arXiv)

While high resolution images contain semantically more useful information than their lower resolution counterparts, processing them is computationally more expensive, and in some applications, e.g. remote sensing, they can be much more expensive to acquire. For these reasons, it is desirable to develop an automatic method to selectively use high resolution data when necessary while maintaining accuracy and reducing acquisition/run-time cost. In this direction, we propose PatchDrop a reinforcement learning approach to dynamically identify when and where to use/acquire high resolution data conditioned on the paired, cheap, low resolution images. We conduct experiments on CIFAR10, CIFAR100, ImageNet and fMoW datasets where we use significantly less high resolution data while maintaining similar accuracy to models which use full high resolution images.