Multimodal Model for Computational Pathology:Representation Learning and Image Compression
Not on the map. Why?
Abstract (arXiv)
Whole slide imaging (WSI) has transformed digital pathology by enabling computational analysis of gigapixel histopathology images. Recent foundation model advances have accelerated progress in computational pathology, facilitating joint reasoning across pathology images, clinical reports, and structured data. Despite this progress, challenges remain: the extreme resolution of WSIs creates computational hurdles for visual learning; limited expert annotations constrain supervised approaches; integrating multimodal information while preserving biological interpretability remains difficult; and the opacity of modeling ultra-long visual sequences hinders clinical transparency. This review comprehensively surveys recent advances in multimodal computational pathology. We systematically analyze four research directions: (1) self-supervised representation learning and structure-aware token compression for WSIs; (2) multimodal data generation and augmentation; (3) parameter-efficient adaptation and reasoning-enhanced few-shot learning; and (4) multi-agent collaborative reasoning for trustworthy diagnosis. We specifically examine how token compression enables cross-scale modeling and how multi-agent mechanisms simulate a pathologist's "Chain of Thought" across magnifications to achieve uncertainty-aware evidence fusion. Finally, we discuss open challenges and argue that future progress depends on unified multimodal frameworks integrating high-resolution visual data with clinical and biomedical knowledge to support interpretable and safe AI-assisted diagnosis.
Key facts. Cited by 6 papers in the map (data through 14 Oct 2026). Builds on 61 of its 91 references that are in the map. First posted on arXiv 19 Mar 2026.
Builds on
61 of 91 references.
References · 61
- 2020A Simple Framework for Contrastive Learning of Visual Representationscited by 12,049
- 2023Visual Instruction Tuningcited by 7,869
- 2021Masked Autoencoders Are Scalable Vision Learnerscited by 7,403
- 2025DeepSeek-R1: Incentivizing Reasoning Capability in LLMs via Reinforcement Learningcited by 6,330
- 2023BLIP-2: Bootstrapping Language-Image Pre-training with Frozen Image Encoders and Large Language Modelscited by 6,137
- 2024GPT-4o System Cardcited by 4,588
- 2025Qwen3-VL Technical Reportcited by 2,793
- 2023LLaVA-Med: Training a Large Language-and-Vision Assistant for Biomedicine in One Daycited by 1,305
- 2022Token Merging: Your ViT But Fastercited by 1,021
- 2025OpenAI GPT-5 System Cardcited by 989
- 2020Data Efficient and Weakly Supervised Computational Pathology on Whole Slide Imagescited by 745
- 2021TransMIL: Transformer based Correlated Multiple Instance Learning for Whole Slide Image Classificationcited by 555
38 more
- 2021Adaptive Token Sampling For Efficient Vision Transformerscited by 183
- 2025LightThinker: Thinking Step-by-Step Compressioncited by 104
- 2023Quilt-LLaVA: Visual Instruction Tuning by Extracting Localized Narratives from Open-Source Histopathology Videoscited by 86
- 2024VoCo-LLaMA: Towards Vision Compression with Large Language Modelscited by 79
- 2024Adapting Visual-Language Models for Generalizable Anomaly Detection in Medical Imagescited by 69
- 2024Feature Re-Embedding: Towards Foundation Model-Level Performance in Computational Pathologycited by 67
- 2024SlideChat: A Large Vision-Language Assistant for Whole-Slide Pathology Image Understandingcited by 57
- 2023Multi-modal Gated Mixture of Local-to-Global Experts for Dynamic Image Fusioncited by 54
- 2023Text-guided Foundation Model Adaptation for Pathological Image Classificationcited by 47
- 20242DMamba: Efficient State Space Model for Image Representation with Applications on Giga-Pixel Whole Slide Image Classificationcited by 35
- 2024CPath-Omni: A Unified Multimodal Foundation Model for Patch and Whole Slide Image Analysis in Computational Pathologycited by 34
- 2024Calibrating Multi-modal Representations: A Pursuit of Group Robustness without Annotationscited by 30
- 2025I2MoE: Interpretable Multimodal Interaction-aware Mixture-of-Expertscited by 30
- 2025Stronger-MAS: Multi-Agent Reinforcement Learning for Collaborative LLMscited by 28
- 2023Segmentation and Vascular Vectorization for Coronary Artery by Geometry-based Cascaded Neural Networkcited by 21
- 2025AdaptVision: Efficient Vision-Language Models via Adaptive Visual Acquisitioncited by 21
- 2024FOCUS: Knowledge-enhanced Adaptive Visual Compression for Few-shot Whole Slide Image Classificationcited by 20
- 2025Scalable Generation of Spatial Transcriptomics from Histology Images via Whole-Slide Flow Matchingcited by 19
- 2024PLUTO: Pathology-Universal Transformercited by 17
- 2024MedViLaM: A multimodal large language model with advanced generalizability and explainability for medical data understanding and generationcited by 17
- 2026TC-SSA: Token Compression via Semantic Slot Aggregation for Gigapixel Pathology Reasoningcited by 17
- 2024One Leaf Reveals the Season: Occlusion-Based Contrastive Learning with Semantic-Aware Views for Efficient Visual Representationcited by 16
- 2025Fewer Tokens, Greater Scaling: Self-Adaptive Visual Bases for Efficient and Expansive Representation Learningcited by 16
- 2023Learning A Multi-Task Transformer Via Unified And Customized Instruction Tuning For Chest Radiograph Interpretationcited by 14
- 2024Interpretable Vision-Language Survival Analysis with Ordinal Inductive Bias for Computational Pathologycited by 14
- 2025Robust Multimodal Survival Prediction with the Latent Differentiation Conditional Variational AutoEncodercited by 13
- 2024TCFormer: Visual Recognition via Token Clustering Transformercited by 12
- 2024Instruction Tuning-free Visual Token Complement for Multimodal LLMscited by 12
- 2024A foundation model for generalizable disease diagnosis in chest X-ray imagescited by 12
- 2024Rethinking Transformer for Long Contextual Histopathology Whole Slide Image Analysiscited by 12
- 2024Free Lunch in Pathology Foundation Model: Task-specific Model Adaptation with Concept-Guided Feature Enhancementcited by 11
- 2026Fine-R1: Make Multi-modal LLMs Excel in Fine-Grained Visual Recognition by Chain-of-Thought Reasoningcited by 11
- 2026The Model Knows Which Tokens Matter: Automatic Token Selection via Noise Gatingcited by 11
- 2026ZeroSense:How Vision matters in Long Context Compressioncited by 11
- 2026Efficient Chest X-ray Representation Learning via Semantic-Partitioned Contrastive Learningcited by 10
- 2025Self-Disentanglement and Re-Composition for Cross-Domain Few-Shot Segmentationcited by 9
- 2024WSI-SAM: Multi-resolution Segment Anything Model (SAM) for histopathology whole-slide imagescited by 7
- 2024Cost-effective Instruction Learning for Pathology Vision and Language Analysiscited by 7
50 most cited of 61 All 61 (JSON)
Built on by
Citing papers per year: 2026: 6.
6 papers.
- 2026Beyond Surrogate Gradients: Fully Differentiable Token Pruning for Vision-Language Modelscited by 7
- 2026Stepwise Token Selection for Efficient Multimodal Large Language Modelscited by 5
- 2026Learnable Token Sparsification for Efficient Gigapixel Whole Slide Image Reasoningcited by 4
- 2026PathSelect: Sequential Token Selection for Whole Slide Pathologycited by 2
- 2026DiffPrune: differentiable information throttling for token pruning in vision-language modelscited by 1
- 2026StepPrune: Adaptive Sequential Visual Token Selection across Multimodal Large Language Modelscited by 0
- 2026StepPrune: Adaptive Sequential Visual Token Selection across Multimodal Large Language Models12 Sep 2026
- 2026DiffPrune: differentiable information throttling for token pruning in vision-language models3 Aug 2026
- 2026PathSelect: Sequential Token Selection for Whole Slide Pathology26 Jul 2026
- 2026Stepwise Token Selection for Efficient Multimodal Large Language Models14 Jun 2026
- 2026Learnable Token Sparsification for Efficient Gigapixel Whole Slide Image Reasoning7 Jun 2026
- 2026Beyond Surrogate Gradients: Fully Differentiable Token Pruning for Vision-Language Models27 May 2026
Similar, not cited
Close in wording, no citation either way
- 2025PolyPath: Adapting a Large Multimodal Model for Multi-slide Pathology Report Generation
- 2026MLLM-HWSI: A Multimodal Large Language Model for Hierarchical Whole Slide Image Understanding
- 2026Benchmarking Vision-Language Models for Automated Pathology Diagnosis and Report Generation
- 2025WSI-Agents: A Collaborative Multi-Agent System for Multi-Modal Whole Slide Image Analysis
- 2025Evidence-based diagnostic reasoning with multi-agent copilot for human pathology
- 2024PathAlign: A vision-language model for whole slide images in histopathology
- 2025MPath: Multimodal Pathology Report Generation from Whole Slide Images
- 2025PathVQ: Reforming Computational Pathology Foundation Model for Whole Slide Image Analysis via Vector Quantization