mlsanityData to 14 Oct 2026

SlideChat: A Large Vision-Language Assistant for Whole-Slide Pathology Image Understanding

arXiv 2410.11761 ↗First posted 15 Oct 2024cs.CV cs.AIcited by 57

On the map: Multimodal LLMs / vision-language models → Medical vision-language models

Abstract (arXiv)

Despite the progress made by multimodal large language models (MLLMs) in computational pathology, they remain limited by a predominant focus on patch-level analysis, missing essential contextual information at the whole-slide level. The lack of large-scale instruction datasets and the gigapixel scale of whole slide images (WSIs) pose significant developmental challenges. In this paper, we present SlideChat, the first vision-language assistant capable of understanding gigapixel whole-slide images, exhibiting excellent multimodal conversational capability and response complex instruction across diverse pathology scenarios. To support its development, we created SlideInstruction, the largest instruction-following dataset for WSIs consisting of 4.2K WSI captions and 176K VQA pairs with multiple categories. Furthermore, we propose SlideBench, a multimodal benchmark that incorporates captioning and VQA tasks to assess SlideChat's capabilities in varied clinical settings such as microscopy, diagnosis. Compared to both general and specialized MLLMs, SlideChat exhibits exceptional capabilities achieving state-of-the-art performance on 18 of 22 tasks. For example, it achieved an overall accuracy of 81.17% on SlideBench-VQA (TCGA), and 54.15% on SlideBench-VQA (BCNB). Our code, data, and model is publicly accessible at https://uni-medical.github.io/SlideChat.github.io.

Key facts. Cited by 57 papers in the map (data through 14 Oct 2026). Placed in Medical vision-language models, one of 16 directions in Multimodal LLMs / vision-language models. Builds on 21 of its 36 references that are in the map. First posted on arXiv 15 Oct 2024.

Builds on

21 of 36 references.

Closest ancestors · 3

Same direction, newest first

  1. 2024PathAlign: A vision-language model for whole slide images in histopathology27 Jun
  2. 2023LLaVA-Med: Training a Large Language-and-Vision Assistant for Biomedicine in One Day1 Jun
  3. 2023PathAsst: A Generative Foundation AI Assistant Towards Artificial General Intelligence of Pathology24 May

Other references · 18

  1. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,744
  2. 2023GPT-4 Technical Reportcited by 18,396
  3. 2023Qwen Technical Reportcited by 3,344
  4. 2024Qwen2 Technical Reportcited by 2,120
  5. 2024Phi-3 Technical Report: A Highly Capable Language Model Locally on Your Phonecited by 1,917
  6. 2024InternLM2 Technical Reportcited by 397
  7. 2023LongNet: Scaling Transformers to 1,000,000,000 Tokenscited by 204
  8. 2023Artificial Intelligence for Digital and Computational Pathologycited by 108
  9. 2020Deep Learning-based Computational Pathology Predicts Origins for Cancers of Unknown Primarycited by 89
  10. 2023Quilt-LLaVA: Visual Instruction Tuning by Extracting Localized Narratives from Open-Source Histopathology Videoscited by 86
  11. 2023Cross-Modal Translation and Alignment for Survival Analysiscited by 70
  12. 2024A Multimodal Knowledge-enhanced Whole-slide Pathology Foundation Modelcited by 59
6 more
  1. 2021Predicting Axillary Lymph Node Metastasis in Early Breast Cancer Using Deep Learning on Primary Tumor Biopsy Slidescited by 43
  2. 2024HistGen: Histopathology Report Generation via Local-Global Feature Encoding and Cross-modal Context Interactioncited by 43
  3. 2024WSI-VQA: Interpreting Whole Slide Images by Generative Visual Question Answeringcited by 34
  4. 2023Computational Pathology: A Survey Review and The Way Forwardcited by 30
  5. 2024Generalizable Whole Slide Image Classification with Fine-Grained Visual-Semantic Interactioncited by 27
  6. 2023HVTSurv: Hierarchical Vision Transformer for Patient-Level Survival Prediction from Whole Slide Imagecited by 14

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2024: 1; 2025: 22; 2026: 34.

57 papers.

Sort:

By area:

  1. 2024WSI-LLaVA: A Multimodal Large Language Model for Whole Slide Imagecited by 40
  2. 2025PathFinder: A Multi-Modal Multi-Agent System for Medical Diagnostic Decision-Making Applied to Histopathologycited by 39
  3. 2025CPathAgent: An Agent-based Foundation Model for Interpretable High-Resolution Pathology Image Analysis Mimicking Pathologists' Diagnostic Logiccited by 29
  4. 2026GameDevBench: Evaluating Agentic Capabilities Through Game Developmentcited by 28
  5. 2025Evidence-based diagnostic reasoning with multi-agent copilot for human pathologycited by 27
  6. 2025MedGround-R1: Advancing Medical Image Grounding via Spatial-Semantic Rewarded Group Relative Policy Optimizationcited by 23
  7. 2025A Versatile Pathology Co-pilot via Reasoning Enhanced Multimodal Large Language Modelcited by 12
  8. 2025UniMedVL: Unifying Medical Multimodal Understanding and Generation through Observation-Knowledge-Analysiscited by 12
  9. 2025Multi-Modal Foundation Models for Computational Pathology: A Surveycited by 9
  10. 2025WSI-Agents: A Collaborative Multi-Agent System for Multi-Modal Whole Slide Image Analysiscited by 7
  11. 2025Navigating Gigapixel Pathology Images with Large Multimodal Modelscited by 7
  12. 2026PathReasoner-R1: Instilling Structured Reasoning into Pathology Vision-Language Model via Knowledge-Guided Policy Optimizationcited by 7
38 more
  1. 2026PathNavigate: A Training-Free Pathology Agent with Surprise-Guided Scan and Shared Slide Memory for Whole-Slide Image VQAcited by 7
  2. 2025Efficient Whole Slide Pathology VQA via Token Compressioncited by 6
  3. 2025LoC-Path: Learning to Compress for Pathology Multimodal Large Language Modelscited by 6
  4. 2026Multimodal Model for Computational Pathology:Representation Learning and Image Compressioncited by 6
  5. 2025Content Generation Models in Computational Pathology: A Comprehensive Survey on Methods, Applications, and Challengescited by 5
  6. 2025TeamPath: Building MultiModal Pathology Experts with Reasoning AI Copilotscited by 5
  7. 2026Act Like a Pathologist: Tissue-Aware Whole Slide Image Reasoningcited by 5
  8. 2025PathFLIP: Fine-grained Language-Image Pretraining for Versatile Computational Pathologycited by 4
  9. 2025Beyond Pixel Simulation: Pathology Image Generation via Diagnostic Semantic Tokens and Prototype Controlcited by 4
  10. 2026QCAgent: An agentic framework for quality-controllable pathology report generation from whole slide imagecited by 4
  11. 2025Domain Specific Benchmarks for Evaluating Multimodal Large Language Modelscited by 3
  12. 2026MLLM-HWSI: A Multimodal Large Language Model for Hierarchical Whole Slide Image Understandingcited by 3
  13. 2026Enhancing Pathological VLMs with Cross-scale Reasoningcited by 3
  14. 2025Benchmarking Foundation Models for Mitotic Figure Classificationcited by 2
  15. 2025Universal and Transferable Attacks on Pathology Foundation Modelscited by 2
  16. 2025Democratising Pathology Co-Pilots: An Open Pipeline and Dataset for Whole-Slide Vision-Language Modellingcited by 2
  17. 2026PulseMind: A Multi-Modal Medical Model for Real-World Clinical Diagnosiscited by 2
  18. 2026NuNext: Reframing Nucleus Detection as Next-Point Detectioncited by 2
  19. 2026PathSelect: Sequential Token Selection for Whole Slide Pathologycited by 2
  20. 2026Beyond Relevance: Bayesian Evidence Acquisition for Agentic Whole-Slide Image Reasoningcited by 2
  21. 2025Reverse Physician-AI Relationship: Full-process Clinical Diagnosis Driven by a Large Language Modelcited by 1
  22. 2026CTIS-QA: Clinical Template-Informed Slide-level Question Answering for Pathologycited by 1
  23. 2026Hepato-LLaVA: An Expert MLLM with Sparse Topo-Pack Attention for Hepatocellular Pathology Analysis on Whole Slide Imagescited by 1
  24. 2026PathMoE: Interpretable Multimodal Interaction Experts for Pediatric Brain Tumor Classificationcited by 1
  25. 2026PathMem: Toward Cognition-Aligned Memory Transformation for Pathology MLLMscited by 1
  26. 2026Auditing Data Leakage in Whole-Slide Image Multimodal Benchmarkscited by 1
  27. 2026Agentic Visual Reasoning in Whole-Slide Pathology Images via Active Perceptioncited by 1
  28. 2026SlideBank: A Persistent Hierarchical Evidence Bank for Consistent Whole-Slide Reasoningcited by 1
  29. 2025Contrastive Cross-Bag Augmentation for Multiple Instance Learning-based Whole Slide Image Classificationcited by 0
  30. 2025HyperST: Hierarchical Hyperbolic Learning for Spatial Transcriptomics Predictioncited by 0
  31. 2026PPGL-Swarm: Integrated Multimodal Risk Stratification and Hereditary Syndrome Detection in Pheochromocytoma and Paragangliomacited by 0
  32. 2026PBSBench: A Multi-Level Vision-Language Framework and Benchmark for Hematopathology Whole Slide Image Interpretationcited by 0
  33. 2026How Seemingly Inconsequential Design Choices Dictate Performance of LLMs in Pathologycited by 0
  34. 2026Semantic-Anchored Evidential Fusion for Domain-Robust Whole-Slide Survival Analysiscited by 0
  35. 2026HERO: Hypothesis-Driven Evidence Retrieval from Omics for Multi-Task Breast Cancer Analysiscited by 0
  36. 2026Mitigating Batch Effects in Histopathology via Language-Mediated Robust Embedding Generationcited by 0
  37. 2026TaxoMIL: Taxonomy-Constrained Learning for Hierarchical Whole Slide Image Analysiscited by 0
  38. 2026Towards Reliable Stain Transfer: An Iterative Data-Model Co-Optimization Framework Based on Multimodal Expert-Guided Assessmentcited by 0

50 most cited of 57 All 57 (JSON)

Similar, not cited

Close in wording, no citation either way

  1. 2026Simple Token-Efficient Vision-Language Model for Case-level Pathology Synoptic Report Generation
  2. 2025LanGuSTE: Language-Guided Coarse-to-Fine Patch Selection for Efficient Whole Slide Image Analysis
  3. 2026Spatial Message Passing in Language Space for Pathology Image Interpretation
  4. 2025Investigating Zero-Shot Diagnostic Pathology in Vision-Language Models with Efficient Prompt Design
  5. 2024CPath-Omni: A Unified Multimodal Foundation Model for Patch and Whole Slide Image Analysis in Computational Pathology
  6. 2025PolyPath: Adapting a Large Multimodal Model for Multi-slide Pathology Report Generation
  7. 2025MPath: Multimodal Pathology Report Generation from Whole Slide Images
  8. 2025VLEER: Vision and Language Embeddings for Explainable Whole Slide Image Representation