# A Multi-Dataset Benchmark of Multiple Instance Learning for 3D Neuroimage Classification

arXiv: https://arxiv.org/abs/2604.26807 · HTML: https://mlsanity.com/paper/2604.26807/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Builds on 20 of its 54 references that are in the map.
- First posted on arXiv 29 Apr 2026.

## Builds on (most cited first, 20 of 20)

- [An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale](https://mlsanity.com/paper/2010.11929/) (2020)
- [Simple and Scalable Predictive Uncertainty Estimation using Deep Ensembles](https://mlsanity.com/paper/1612.01474/) (2016)
- [A ConvNet for the 2020s](https://mlsanity.com/paper/2201.03545/) (2022)
- [A Closer Look at Spatiotemporal Convolutions for Action Recognition](https://mlsanity.com/paper/1711.11248/) (2017)
- [Attention-based Deep Multiple Instance Learning](https://mlsanity.com/paper/1802.04712/) (2018)
- [TransMIL: Transformer based Correlated Multiple Instance Learning for Whole Slide Image Classification](https://mlsanity.com/paper/2106.00908/) (2021)
- [Convolutional Neural Networks for Classification of Alzheimer's Disease: Overview and Reproducible Evaluation](https://mlsanity.com/paper/1904.07773/) (2019)
- [QoQ-Med: Building Multimodal Clinical Foundation Models with Domain-Aware GRPO Training](https://mlsanity.com/paper/2506.00711/) (2025)
- [Attention Correctness in Neural Image Captioning](https://mlsanity.com/paper/1605.09553/) (2016)
- [Deep Multi-instance Networks with Sparse Label Assignment for Whole Mammogram Classification](https://mlsanity.com/paper/1612.05968/) (2016)
- [A Contrast-Adaptive Method for Simultaneous Whole-Brain and Lesion Segmentation in Multiple Sclerosis](https://mlsanity.com/paper/2005.05135/) (2020)
- [CLIMB: Data Foundations for Large Scale Multimodal Clinical Foundation Models](https://mlsanity.com/paper/2503.07667/) (2025)
- [On the design of convolutional neural networks for automatic detection of Alzheimer's disease](https://mlsanity.com/paper/1911.03740/) (2019)
- [Reproducibility in Multiple Instance Learning: A Case For Algorithmic Unit Tests](https://mlsanity.com/paper/2310.17867/) (2023)
- [Sm: enhanced localization in Multiple Instance Learning for medical imaging classification](https://mlsanity.com/paper/2410.03276/) (2024)
- [An Open-Source Tool for Longitudinal Whole-Brain and White Matter Lesion Segmentation](https://mlsanity.com/paper/2207.04534/) (2022)
- [Systematic comparison of semi-supervised and self-supervised learning for medical image classification](https://mlsanity.com/paper/2307.08919/) (2023)
- [Benchmarking CNN on 3D Anatomical Brain MRI: Architectures, Data Augmentation and Deep Ensemble Learning](https://mlsanity.com/paper/2106.01132/) (2021)
- [A Probabilistic Method to Predict Classifier Accuracy on Larger Datasets given Small Pilot Data](https://mlsanity.com/paper/2311.18025/) (2023)
- [Smooth Attention for Deep Multiple Instance Learning: Application to CT Intracranial Hemorrhage Detection](https://mlsanity.com/paper/2307.09457/) (2023)

## Built on by (most cited first, 1 of 1)

- [Normal Guidance is what Attention Needs](https://mlsanity.com/paper/2605.27306/) (2026)

## Abstract (arXiv)

Despite being resource-intensive to train, 3D convolutional neural networks (CNNs) have been the standard approach to classify CT and MRI scans. Recent work suggests that deep multiple instance learning (MIL) may be a more efficient alternative for 3D brain scans, especially when the pre-trained image encoder used to embed each 2D slice is frozen and only the pooling operation and classifier are trained. In this paper, we provide a systematic comparison of simple MIL, attention-based MIL, 3D CNNs, and 3D ViTs across three CT and four MRI datasets, including two large datasets of at least 10,000 scans. Our goal is to help resource-constrained practitioners understand which neural networks work well for 3D neuroimages and why. We further compare design choices for attention-based MIL, including different encoders, pooling operations, and architectural orderings. We find that simple mean pooling MIL, without any learnable attention, matches or outperforms recent MIL or 3D CNN alternatives on 4 of 6 moderate-sized tasks. This baseline remains competitive on two large datasets while being 25x faster to train. To explain mean pooling's success, we examine per-slice attention quality and a semi-synthetic dataset where we can derive the best possible classifier via a Bayes estimator. This analysis reveals the limits of existing MIL approaches and suggests routes for future improvements.