# Patch Gradient Descent: Training Neural Networks on Very Large Images

arXiv: https://arxiv.org/abs/2301.13817 · HTML: https://mlsanity.com/paper/2301.13817/

## Key facts

- Cited by 2 papers in the map (data through 14 Oct 2026).
- Builds on 17 of its 40 references that are in the map.
- First posted on arXiv 31 Jan 2023.

## Builds on (most cited first, 17 of 17)

- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [MobileNetV2: Inverted Residuals and Linear Bottlenecks](https://mlsanity.com/paper/1801.04381/) (2018)
- [A Survey of Convolutional Neural Networks: Analysis, Applications, and Prospects](https://mlsanity.com/paper/2004.02806/) (2020)
- [Scaling Vision Transformers to Gigapixel Images via Hierarchical Self-Supervised Learning](https://mlsanity.com/paper/2206.02647/) (2022)
- [A Survey of the Recent Architectures of Deep Convolutional Neural Networks](https://mlsanity.com/paper/1901.06032/) (2019)
- [Differentiable Patch Selection for Image Recognition](https://mlsanity.com/paper/2104.03059/) (2021)
- [Neural Image Compression for Gigapixel Histopathology Image Analysis](https://mlsanity.com/paper/1811.02840/) (2018)
- [Cluster-to-Conquer: A Framework for End-to-End Multi-Instance Learning for Whole Slide Image Classification](https://mlsanity.com/paper/2103.10626/) (2021)
- [Detecting Cancer Metastases on Gigapixel Pathology Images](https://mlsanity.com/paper/1703.02442/) (2017)
- [Streaming convolutional neural networks for end-to-end learning with multi-megapixel images](https://mlsanity.com/paper/1911.04432/) (2019)
- [Processing Megapixel Images with Deep Attention-Sampling Models](https://mlsanity.com/paper/1905.03711/) (2019)
- [Hard-Attention for Scalable Image Classification](https://mlsanity.com/paper/2102.10212/) (2021)
- [Extending Unsupervised Neural Image Compression With Supervised Multitask Learning](https://mlsanity.com/paper/2004.07041/) (2020)
- [Resolution-Based Distillation for Efficient Histology Image Classification](https://mlsanity.com/paper/2101.04170/) (2021)
- [Gigapixel Histopathological Image Analysis using Attention-based Neural Networks](https://mlsanity.com/paper/2101.09992/) (2021)
- [UltraMNIST Classification: A Benchmark to Train CNNs for Very Large Images](https://mlsanity.com/paper/2206.12681/) (2022)
- [Deep Learning-Based Sparse Whole-Slide Image Analysis for the Diagnosis of Gastric Intestinal Metaplasia](https://mlsanity.com/paper/2201.01449/) (2022)

## Built on by (most cited first, 2 of 2)

- [Resource Efficient Perception for Vision Systems](https://mlsanity.com/paper/2405.07166/) (2024)
- [Stride Independent Patching for Deep Learning](https://mlsanity.com/paper/2610.12216/) (2026)

## Abstract (arXiv)

Traditional CNN models are trained and tested on relatively low resolution images (<300 px), and cannot be directly operated on large-scale images due to compute and memory constraints. We propose Patch Gradient Descent (PatchGD), an effective learning strategy that allows to train the existing CNN architectures on large-scale images in an end-to-end manner. PatchGD is based on the hypothesis that instead of performing gradient-based updates on an entire image at once, it should be possible to achieve a good solution by performing model updates on only small parts of the image at a time, ensuring that the majority of it is covered over the course of iterations. PatchGD thus extensively enjoys better memory and compute efficiency when training models on large scale images. PatchGD is thoroughly evaluated on two datasets - PANDA and UltraMNIST with ResNet50 and MobileNetV2 models under different memory constraints. Our evaluation clearly shows that PatchGD is much more stable and efficient than the standard gradient-descent method in handling large images, and especially when the compute memory is limited.