# Learnable Token Sparsification for Efficient Gigapixel Whole Slide Image Reasoning

arXiv: https://arxiv.org/abs/2606.08641 · HTML: https://mlsanity.com/paper/2606.08641/

## Key facts

- Cited by 4 papers in the map (data through 14 Oct 2026).
- Placed in Multimodal reasoning, one of 17 directions in LLM reasoning.
- Builds on 18 of its 52 references that are in the map.
- First posted on arXiv 7 Jun 2026.

## Placed in

- LLM reasoning / [Multimodal reasoning](https://mlsanity.com/lr/multimodal-reasoning/)
- Multimodal LLMs / vision-language models / [Visual token pruning and efficiency](https://mlsanity.com/mllm/visual-token-pruning-and-efficiency/)

## Builds on (most cited first, 18 of 18)

- [GPT-4o System Card](https://mlsanity.com/paper/2410.21276/) (2024)
- [Categorical Reparameterization with Gumbel-Softmax](https://mlsanity.com/paper/1611.01144/) (2016)
- [Estimating or Propagating Gradients Through Stochastic Neurons for Conditional Computation](https://mlsanity.com/paper/1308.3432/) (2013)
- [GSCo: Towards Generalizable AI in Medicine via Generalist-Specialist Collaboration](https://mlsanity.com/paper/2404.15127/) (2024)
- [A Survey of Token Compression for Efficient Multimodal Large Language Models](https://mlsanity.com/paper/2507.20198/) (2025)
- [MedViLaM: A multimodal large language model with advanced generalizability and explainability for medical data understanding and generation](https://mlsanity.com/paper/2409.19684/) (2024)
- [TC-SSA: Token Compression via Semantic Slot Aggregation for Gigapixel Pathology Reasoning](https://mlsanity.com/paper/2603.01143/) (2026)
- [Fewer Tokens, Greater Scaling: Self-Adaptive Visual Bases for Efficient and Expansive Representation Learning](https://mlsanity.com/paper/2511.19515/) (2025)
- [A foundation model for generalizable disease diagnosis in chest X-ray images](https://mlsanity.com/paper/2410.08861/) (2024)
- [The Model Knows Which Tokens Matter: Automatic Token Selection via Noise Gating](https://mlsanity.com/paper/2603.07135/) (2026)
- [ZeroSense:How Vision matters in Long Context Compression](https://mlsanity.com/paper/2603.11846/) (2026)
- [Efficient Chest X-ray Representation Learning via Semantic-Partitioned Contrastive Learning](https://mlsanity.com/paper/2603.07113/) (2026)
- [LearnPruner: Rethinking Attention-based Token Pruning in Vision Language Models](https://mlsanity.com/paper/2604.23950/) (2026)
- [Beyond Surrogate Gradients: Fully Differentiable Token Pruning for Vision-Language Models](https://mlsanity.com/paper/2605.28051/) (2026)
- [LoC-Path: Learning to Compress for Pathology Multimodal Large Language Models](https://mlsanity.com/paper/2512.05391/) (2025)
- [Multimodal Model for Computational Pathology:Representation Learning and Image Compression](https://mlsanity.com/paper/2603.18660/) (2026)
- [A unified multi-task framework enables interpretable chest radiograph analysis](https://mlsanity.com/paper/2606.03417/) (2026)
- [XrayClaw: Cooperative-Competitive Multi-Agent Alignment for Trustworthy Chest X-ray Diagnosis](https://mlsanity.com/paper/2604.02695/) (2026)

## Built on by (most cited first, 4 of 4)

- [PathSelect: Sequential Token Selection for Whole Slide Pathology](https://mlsanity.com/paper/2607.23631/) (2026)
- [DiffPrune: differentiable information throttling for token pruning in vision-language models](https://mlsanity.com/paper/2608.01985/) (2026)
- [StepPrune: Adaptive Sequential Visual Token Selection across Multimodal Large Language Models](https://mlsanity.com/paper/2609.13804/) (2026)
- [MedTokenBudget: Lesion-Preserving Token Routing for Dermoscopic Image Classification](https://mlsanity.com/paper/2609.30613/) (2026)

## Abstract (arXiv)

The processing of gigapixel whole slide images within vision language models faces a major difficulty due to an excessive number of visual tokens. Existing solutions typically rely on spatial downsampling or heuristic pruning strategies that operate without training, and these methods often discard subtle but clinically meaningful patterns because pathological evidence is scattered irregularly across the tissue. To overcome this limitation, we reformulate token reduction in whole slide images as a trainable sparsification problem, allowing the model to learn an optimal selection strategy instead of following fixed heuristics. We propose a decoupled routing architecture. To enable gradient propagation through the nondifferentiable pruning operation during training, we introduce a component called SparseLearn. This component uses a variance-preserving noise gate that regulates the information flow of each patch via a differentiable Soft Top-K operator, together with a diagonal attention denoiser that recovers perturbed representations without leaking spatial information. At inference time, the SparseLearn module is entirely discarded, and the trained scorer applies a deterministic Hard Top-K operator to keep only the highest scoring 32 tokens, incurring no extra computation. By compressing the visual sequence down to a sparse set of just 32 tokens, which represents as little as 0.78% of the original length, our framework achieves 73.32% overall accuracy on SlideBench (TCGA), consistently surpassing sampling-based baselines and general-purpose vision language models. It also demonstrates strong zero shot generalization on SlideBench (BCNB) and WSI VQA*. By resolving the visual context bottleneck and preventing the dilution of sparse diagnostic evidence, this work provides a highly efficient paradigm for end to end gigapixel whole slide image reasoning.