mlsanityData to 14 Oct 2026

Iterative Patch Selection for High-Resolution Image Recognition

arXiv 2210.13007 ↗First posted 24 Oct 2022cs.CV cs.LG eess.IVcited by 14

Not on the map. Why?

Abstract (arXiv)

High-resolution images are prevalent in various applications, such as autonomous driving and computer-aided diagnosis. However, training neural networks on such images is computationally challenging and easily leads to out-of-memory errors even on modern GPUs. We propose a simple method, Iterative Patch Selection (IPS), which decouples the memory usage from the input size and thus enables the processing of arbitrarily large images under tight hardware constraints. IPS achieves this by selecting only the most salient patches, which are then aggregated into a global representation for image recognition. For both patch selection and aggregation, a cross-attention based transformer is introduced, which exhibits a close connection to Multiple Instance Learning. Our method demonstrates strong performance and has wide applicability across different domains, training regimes and image sizes while using minimal accelerator memory. For example, we are able to finetune our model on whole-slide images consisting of up to 250k patches (>16 gigapixels) with only 5 GB of GPU VRAM at a batch size of 16.

Key facts. Cited by 14 papers in the map (data through 14 Oct 2026). Builds on 23 of its 46 references that are in the map. First posted on arXiv 24 Oct 2022.

Builds on

23 of 46 references.

References · 23

  1. 2017Attention Is All You Needcited by 60,361
  2. 2015Deep Residual Learning for Image Recognitioncited by 52,096
  3. 2014ImageNet Large Scale Visual Recognition Challengecited by 8,498
  4. 2020End-to-End Object Detection with Transformerscited by 7,414
  5. 2020Bootstrap your own latent: A new approach to self-supervised Learningcited by 4,750
  6. 2017Outrageously Large Neural Networks: The Sparsely-Gated Mixture-of-Experts Layercited by 3,509
  7. 2019Energy and Policy Considerations for Deep Learning in NLPcited by 1,254
  8. 2018Attention-based Deep Multiple Instance Learningcited by 1,095
  9. 2021CrossViT: Cross-Attention Multi-Scale Vision Transformer for Image Classificationcited by 806
  10. 2021Scaling Vision with Sparse Mixture of Expertscited by 747
  11. 2020Data Efficient and Weakly Supervised Computational Pathology on Whole Slide Imagescited by 745
  12. 2021Perceiver IO: A General Architecture for Structured Inputs & Outputscited by 629
11 more
  1. 2020Dual-stream Multiple Instance Learning Network for Whole Slide Image Classification with Self-supervised Contrastive Learningcited by 478
  2. 2022Scaling Vision Transformers to Gigapixel Images via Hierarchical Self-Supervised Learningcited by 359
  3. 2016Deep Learning for Identifying Metastatic Breast Cancercited by 93
  4. 2021Differentiable Patch Selection for Image Recognitioncited by 68
  5. 2020Weakly supervised multiple instance learning histopathological tumor segmentationcited by 46
  6. 2020Self-Supervision Closes the Gap Between Weak and Strong Supervision in Histologycited by 43
  7. 2019Streaming convolutional neural networks for end-to-end learning with multi-megapixel imagescited by 41
  8. 2019Processing Megapixel Images with Deep Attention-Sampling Modelscited by 29
  9. 2021Learned Queries for Efficient Local Attentioncited by 27
  10. 2021Efficient Classification of Very Large Images with Tiny Objectscited by 17
  11. 2018Terabyte-scale Deep Multiple Instance Learning for Classification and Localization in Pathologycited by 16

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2023: 3; 2024: 5; 2025: 3; 2026: 3.

14 papers.

Sort:

By area:

  1. 2023Visual Language Pretrained Multiple Instance Zero-Shot Transfer for Histopathology Imagescited by 134
  2. 2023Attention-Challenging Multiple Instance Learning for Whole Slide Image Classificationcited by 61
  3. 2024Token Cropr: Faster ViTs for Quite a Few Taskscited by 19
  4. 2024Rethinking Transformer for Long Contextual Histopathology Whole Slide Image Analysiscited by 12
  5. 2025LookWhere? Efficient Visual Recognition by Learning Where to Look and What to See from Self-Supervisioncited by 5
  6. 2025PathVQ: Reforming Computational Pathology Foundation Model for Whole Slide Image Analysis via Vector Quantizationcited by 3
  7. 2025A Hidden Stumbling Block in Generalized Category Discovery: Distracted Attentioncited by 3
  8. 2026Head-Aware Visual Cropping: Enhancing Fine-Grained VQA with Attention-Guided Subimagecited by 2
  9. 2023Dual-Query Multiple Instance Learning for Dynamic Meta-Embedding based Tumor Classificationcited by 1
  10. 2024Resource Efficient Perception for Vision Systemscited by 0
  11. 2024EPS: Efficient Patch Sampling for Video Overfitting in Deep Super-Resolution Model Trainingcited by 0
  12. 2024On the Generalizability of Iterative Patch Selection for Memory-Efficient High-Resolution Image Classificationcited by 0
2 more
  1. 2026LookThere! Sparse Vision by Reinforced Selectioncited by 0
  2. 2026Stride Independent Patching for Deep Learningcited by 0