# Processing Megapixel Images with Deep Attention-Sampling Models

arXiv: https://arxiv.org/abs/1905.03711 · HTML: https://mlsanity.com/paper/1905.03711/

## Key facts

- Cited by 29 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 3 May 2019.

## Built on by (most cited first, 29 of 29)

- [Artificial Intelligence for Digital and Computational Pathology](https://mlsanity.com/paper/2401.06148/) (2023)
- [Differentiable Patch Selection for Image Recognition](https://mlsanity.com/paper/2104.03059/) (2021)
- [Multi-task Learning with Attention for End-to-end Autonomous Driving](https://mlsanity.com/paper/2104.10753/) (2021)
- [Differentiable Zooming for Multiple Instance Learning on Whole-Slide Images](https://mlsanity.com/paper/2204.12454/) (2022)
- [Glance and Focus Networks for Dynamic Visual Recognition](https://mlsanity.com/paper/2201.03014/) (2022)
- [Vision Transformer with Attentive Pooling for Robust Facial Expression Recognition](https://mlsanity.com/paper/2212.05463/) (2022)
- [Learning to Downsample for Segmentation of Ultra-High Resolution Images](https://mlsanity.com/paper/2109.11071/) (2021)
- [Hard-Attention for Scalable Image Classification](https://mlsanity.com/paper/2102.10212/) (2021)
- [Efficient Classification of Very Large Images with Tiny Objects](https://mlsanity.com/paper/2106.02694/) (2021)
- [Iterative Patch Selection for High-Resolution Image Recognition](https://mlsanity.com/paper/2210.13007/) (2022)
- [A Probabilistic Hard Attention Model For Sequentially Observed Scenes](https://mlsanity.com/paper/2111.07534/) (2021)
- [SALISA: Saliency-based Input Sampling for Efficient Video Object Detection](https://mlsanity.com/paper/2204.02397/) (2022)
- [Consistency driven Sequential Transformers Attention Model for Partially Observable Scenes](https://mlsanity.com/paper/2204.00656/) (2022)
- [Resolution-Based Distillation for Efficient Histology Image Classification](https://mlsanity.com/paper/2101.04170/) (2021)
- [Dual Attention Model with Reinforcement Learning for Classification of Histology Whole-Slide Images](https://mlsanity.com/paper/2302.09682/) (2023)
- [GQHAN: A Grover-inspired Quantum Hard Attention Network](https://mlsanity.com/paper/2401.14089/) (2024)
- [No More Sliding Window: Efficient 3D Medical Image Segmentation with Differentiable Top-k Patch Sampling](https://mlsanity.com/paper/2501.10814/) (2025)
- [LookWhere? Efficient Visual Recognition by Learning Where to Look and What to See from Self-Supervision](https://mlsanity.com/paper/2505.18051/) (2025)
- [Efficient subtyping of ovarian cancer histopathology whole slide images using active sampling in multiple instance learning](https://mlsanity.com/paper/2302.08867/) (2023)
- [HistoPerm: A Permutation-Based View Generation Approach for Improving Histopathologic Feature Representation Learning](https://mlsanity.com/paper/2209.06185/) (2022)
- [Patch Gradient Descent: Training Neural Networks on Very Large Images](https://mlsanity.com/paper/2301.13817/) (2023)
- [Scalable and Equitable Math Problem Solving Strategy Prediction in Big Educational Data](https://mlsanity.com/paper/2308.03892/) (2023)
- [mTREE: Multi-Level Text-Guided Representation End-to-End Learning for Whole Slide Image Analysis](https://mlsanity.com/paper/2405.17824/) (2024)
- [Interpretable and Interactive Deep Multiple Instance Learning for Dental Caries Classification in Bitewing X-rays](https://mlsanity.com/paper/2112.09694/) (2021)
- [High-Fidelity Visual Structural Inspections through Transformers and Learnable Resizers](https://mlsanity.com/paper/2210.12175/) (2022)
- [Learning to Perceive in Deep Model-Free Reinforcement Learning](https://mlsanity.com/paper/2301.03730/) (2023)
- [Mastery Guided Non-parametric Clustering to Scale-up Strategy Prediction](https://mlsanity.com/paper/2401.10210/) (2024)
- [Resource Efficient Perception for Vision Systems](https://mlsanity.com/paper/2405.07166/) (2024)
- [On the Generalizability of Iterative Patch Selection for Memory-Efficient High-Resolution Image Classification](https://mlsanity.com/paper/2412.11237/) (2024)

## Abstract (arXiv)

Existing deep architectures cannot operate on very large signals such as megapixel images due to computational and memory constraints. To tackle this limitation, we propose a fully differentiable end-to-end trainable model that samples and processes only a fraction of the full resolution input image. The locations to process are sampled from an attention distribution computed from a low resolution view of the input. We refer to our method as attention sampling and it can process images of several megapixels with a standard single GPU setup. We show that sampling from the attention distribution results in an unbiased estimator of the full model with minimal variance, and we derive an unbiased estimator of the gradient that we use to train our model end-to-end with a normal SGD procedure. This new method is evaluated on three classification tasks, where we show that it allows to reduce computation and memory footprint by an order of magnitude for the same accuracy as classical architectures. We also show the consistency of the sampling that indeed focuses on informative parts of the input images.