Spatial Blindness in Whole-Slide Multiple Instance Learning
Not on the map. Why?
Abstract (arXiv)
Whole-slide MIL models are often called context-aware once graphs, Transform ers, or state-space modules are placed above patch embeddings. We show that this label can be deceptive. On pathology tasks where tissue architecture is part of the diagnostic signal, several strong MIL baselines retain nearly unchanged slide level AUC after patch coordinates are permuted. Their predictions are accurate, but largely compositional. We refer to this failure mode as spatial blindness. Our explanation is optimization-based: dense appearance statistics are learned early under slide-level supervision, leaving weak gradients for sparse spatial relations. ResTopoMIL addresses the issue by first fitting a permutation-invariant prototype histogram and then freezing it while a lightweight graph branch learns the residual under a coordinate-shuffling constraint. The architecture is simple by design; the intervention is in how the spatial branch is trained. Across 9 public WSI bench marks, ResTopoMIL improves classification and survival prediction with 1.15M parameters, restores sensitivity to coordinate perturbation, and gives stronger lo calization evidence on CAMELYON-16.
Key facts. No citing papers in the map yet (data through 7 Oct 2026). Builds on 46 of its 69 references that are in the map. First posted on arXiv 17 May 2026.
Builds on
46 of 69 references.
References · 46
- 2017Attention Is All You Needcited by 59,932
- 2023DINOv2: Learning Robust Visual Features without Supervisioncited by 7,764
- 2021Emerging Properties in Self-Supervised Vision Transformerscited by 6,893
- 2021Efficiently Modeling Long Sequences with Structured State Spacescited by 2,424
- 2018ImageNet-trained CNNs are biased towards texture; increasing shape bias improves accuracy and robustnesscited by 1,430
- 2017Deep Setscited by 1,394
- 2016Unsupervised Learning of Visual Representations by Solving Jigsaw Puzzlescited by 1,379
- 2018Attention-based Deep Multiple Instance Learningcited by 1,077
- 2020Data Efficient and Weakly Supervised Computational Pathology on Whole Slide Imagescited by 738
- 2021TransMIL: Transformer based Correlated Multiple Instance Learning for Whole Slide Image Classificationcited by 549
- 2020Dual-stream Multiple Instance Learning Network for Whole Slide Image Classification with Self-supervised Contrastive Learningcited by 473
- 2021Nystr"omformer: A Nystr"om-Based Algorithm for Approximating Self-Attentioncited by 468
34 more
- 2022Scaling Vision Transformers to Gigapixel Images via Hierarchical Self-Supervised Learningcited by 357
- 2020The Pitfalls of Simplicity Bias in Neural Networkscited by 349
- 2020Gradient Starvation: A Learning Proclivity in Neural Networkscited by 262
- 2022DTFD-MIL: Double-Tier Feature Distillation Multiple Instance Learning for Histopathology Whole Slide Image Classificationcited by 254
- 2022Benchmarking Self-Supervised Learning on Diverse Pathology Datasetscited by 148
- 2023Visual Language Pretrained Multiple Instance Zero-Shot Transfer for Histopathology Imagescited by 132
- 2021BRACS: A Dataset for BReAst Carcinoma Subtyping in H&E Histology Imagescited by 130
- 2021Whole Slide Images are 2D Point Clouds: Context-Aware Survival Prediction using Patch-based Graph Convolutional Networkscited by 124
- 2023Artificial Intelligence for Digital and Computational Pathologycited by 106
- 2023Interventional Bag Multi-Instance Learning On Whole-Slide Pathological Imagescited by 91
- 2024MambaMIL: Enhancing Long Sequence Modeling with Sequence Reordering in Computational Pathologycited by 87
- 2024Morphological Prototyping for Unsupervised Slide Representation Learning in Computational Pathologycited by 70
- 2023Attention-Challenging Multiple Instance Learning for Whole Slide Image Classificationcited by 61
- 2021Hierarchical Graph Representations in Digital Pathologycited by 58
- 2023Multiple Instance Learning Framework with Masked Hard Instance Mining for Whole Slide Image Classificationcited by 58
- 2023A General-Purpose Self-Supervised Model for Computational Pathologycited by 42
- 2020Representation Learning of Histopathology Images using Graph Neural Networkscited by 40
- 2015Classifying and Segmenting Microscopy Images Using Convolutional Multiple Instance Learningcited by 38
- 20242DMamba: Efficient State Space Model for Image Representation with Applications on Giga-Pixel Whole Slide Image Classificationcited by 35
- 2025ViLa-MIL: Dual-scale Vision-Language Multiple Instance Learning for Whole Slide Image Classificationcited by 28
- 2025Do Multiple Instance Learning Models Transfer?cited by 28
- 2023Rethinking Multiple Instance Learning for Whole Slide Image Classification: A Good Instance Classifier is All You Needcited by 21
- 2020Histogram Layers for Texture Analysiscited by 19
- 2023HIGT: Hierarchical Interaction Graph-Transformer for Whole Slide Image Analysiscited by 17
- 2024MSCPT: Few-shot Whole Slide Image Classification with Multi-scale and Context-focused Prompt Tuningcited by 16
- 2024AEM: Attention Entropy Maximization for Multiple Instance Learning based Whole Slide Image Classificationcited by 10
- 2024Queryable Prototype Multiple Instance Learning with Vision-Language Models for Incremental Whole Slide Image Classificationcited by 10
- 2025GECKO: Gigapixel Vision-Concept Contrastive Pretraining in Histopathologycited by 8
- 2024PATHS: A Hierarchical Transformer for Efficient Whole Slide Image Analysiscited by 7
- 2025How Effective Can Dropout Be in Multiple Instance Learning ?cited by 7
- 2022Bag Graph: Multiple Instance Learning using Bayesian Graph Neural Networkscited by 6
- 2023PDL: Regularizing Multiple Instance Learning with Progressive Dropout Layerscited by 6
- 2025Few-Shot Learning from Gigapixel Images via Hierarchical Vision-Language Alignment and Modelingcited by 3
- 2025Slide-Level Prompt Learning with Vision Language Models for Few-Shot Multiple Instance Learning in Histopathologycited by 1
Built on by
Not cited yet.