# End-to-end Multiple Instance Learning with Gradient Accumulation

arXiv: https://arxiv.org/abs/2203.03981 · HTML: https://mlsanity.com/paper/2203.03981/

## Key facts

- Cited by 4 papers in the map (data through 14 Oct 2026).
- Builds on 12 of its 20 references that are in the map.
- First posted on arXiv 8 Mar 2022.

## Builds on (most cited first, 12 of 12)

- [Attention-based Deep Multiple Instance Learning](https://mlsanity.com/paper/1802.04712/) (2018)
- [Training Deep Nets with Sublinear Memory Cost](https://mlsanity.com/paper/1604.06174/) (2016)
- [Data Efficient and Weakly Supervised Computational Pathology on Whole Slide Images](https://mlsanity.com/paper/2004.09666/) (2020)
- [Albumentations: fast and flexible image augmentations](https://mlsanity.com/paper/1809.06839/) (2018)
- [Dual-stream Multiple Instance Learning Network for Whole Slide Image Classification with Self-supervised Contrastive Learning](https://mlsanity.com/paper/2011.08939/) (2020)
- [Not-so-supervised: a survey of semi-supervised, multi-instance, and transfer learning in medical image analysis](https://mlsanity.com/paper/1804.06353/) (2018)
- [Cluster-to-Conquer: A Framework for End-to-End Multi-Instance Learning for Whole Slide Image Classification](https://mlsanity.com/paper/2103.10626/) (2021)
- [Cold Case: The Lost MNIST Digits](https://mlsanity.com/paper/1905.10498/) (2019)
- [Self-Supervision Closes the Gap Between Weak and Strong Supervision in Histology](https://mlsanity.com/paper/2012.03583/) (2020)
- [Streaming convolutional neural networks for end-to-end learning with multi-megapixel images](https://mlsanity.com/paper/1911.04432/) (2019)
- [Detection of prostate cancer in whole-slide images through end-to-end training with image-level labels](https://mlsanity.com/paper/2006.03394/) (2020)
- [An attention-based multi-resolution model for prostate whole slide imageclassification and localization](https://mlsanity.com/paper/1905.13208/) (2019)

## Built on by (most cited first, 4 of 4)

- [TASTY: A Transformer based Approach to Space and Time complexity](https://mlsanity.com/paper/2305.05379/) (2023)
- [Let it shine: Autofluorescence of Papanicolaou-stain improves AI-based cytological oral cancer detection](https://mlsanity.com/paper/2407.01869/) (2024)
- [GriDiT: Factorized Grid-Based Diffusion for Efficient Long Image Sequence Generation](https://mlsanity.com/paper/2512.21276/) (2025)
- [Beyond Imaging: Vision Transformer Digital Twin Surrogates for 3D+T Biological Tissue Dynamics](https://mlsanity.com/paper/2508.15883/) (2025)

## Abstract (arXiv)

Being able to learn on weakly labeled data, and provide interpretability, are two of the main reasons why attention-based deep multiple instance learning (ABMIL) methods have become particularly popular for classification of histopathological images. Such image data usually come in the form of gigapixel-sized whole-slide-images (WSI) that are cropped into smaller patches (instances). However, the sheer size of the data makes training of ABMIL models challenging. All the instances from one WSI cannot be processed at once by conventional GPUs. Existing solutions compromise training by relying on pre-trained models, strategic sampling or selection of instances, or self-supervised learning. We propose a training strategy based on gradient accumulation that enables direct end-to-end training of ABMIL models without being limited by GPU memory. We conduct experiments on both QMNIST and Imagenette to investigate the performance and training time, and compare with the conventional memory-expensive baseline and a recent sampled-based approach. This memory-efficient approach, although slower, reaches performance indistinguishable from the memory-expensive baseline.