mlsanityData to 14 Oct 2026

WSI-VQA: Interpreting Whole Slide Images by Generative Visual Question Answering

arXiv 2407.05603 ↗First posted 8 Jul 2024cs.CV cs.AIcited by 34

Not on the map. Why?

Abstract (arXiv)

Whole slide imaging is routinely adopted for carcinoma diagnosis and prognosis. Abundant experience is required for pathologists to achieve accurate and reliable diagnostic results of whole slide images (WSI). The huge size and heterogeneous features of WSIs make the workflow of pathological reading extremely time-consuming. In this paper, we propose a novel framework (WSI-VQA) to interpret WSIs by generative visual question answering. WSI-VQA shows universality by reframing various kinds of slide-level tasks in a question-answering pattern, in which pathologists can achieve immunohistochemical grading, survival prediction, and tumor subtyping following human-machine interaction. Furthermore, we establish a WSI-VQA dataset which contains 8672 slide-level question-answering pairs with 977 WSIs. Besides the ability to deal with different slide-level tasks, our generative model which is named Wsi2Text Transformer (W2T) outperforms existing discriminative models in medical correctness, which reveals the potential of our model to be applied in the clinical scenario. Additionally, we also visualize the co-attention mapping between word embeddings and WSIs as an intuitive explanation for diagnostic results. The dataset and related code are available at https://github.com/cpystan/WSI-VQA.

Key facts. Cited by 34 papers in the map (data through 14 Oct 2026). Builds on 25 of its 39 references that are in the map. First posted on arXiv 8 Jul 2024.

Builds on

25 of 39 references.

References · 25

  1. 2017Attention Is All You Needcited by 60,361
  2. 2015Deep Residual Learning for Image Recognitioncited by 52,096
  3. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,744
  4. 2020Language Models are Few-Shot Learnerscited by 32,419
  5. 2020An Image is Worth 16x16 Words: Transformers for Image Recognition at Scalecited by 30,186
  6. 2023Visual Instruction Tuningcited by 7,869
  7. 2021Emerging Properties in Self-Supervised Vision Transformerscited by 6,968
  8. 2023LLaVA-Med: Training a Large Language-and-Vision Assistant for Biomedicine in One Daycited by 1,305
  9. 2018Attention-based Deep Multiple Instance Learningcited by 1,095
  10. 2020Domain-Specific Language Model Pretraining for Biomedical Natural Language Processingcited by 1,009
  11. 2019Publicly Available Clinical BERT Embeddingscited by 1,002
  12. 2020Data Efficient and Weakly Supervised Computational Pathology on Whole Slide Imagescited by 745
13 more
  1. 2023The Dawn of LMMs: Preliminary Explorations with GPT-4V(ision)cited by 681
  2. 2020Dual-stream Multiple Instance Learning Network for Whole Slide Image Classification with Self-supervised Contrastive Learningcited by 478
  3. 2022Scaling Vision Transformers to Gigapixel Images via Hierarchical Self-Supervised Learningcited by 359
  4. 2023PMC-VQA: Visual Instruction Tuning for Medical Visual Question Answeringcited by 338
  5. 2022DTFD-MIL: Double-Tier Feature Distillation Multiple Instance Learning for Histopathology Whole Slide Image Classificationcited by 256
  6. 2023PMC-CLIP: Contrastive Language-Image Pre-training using Biomedical Documentscited by 212
  7. 2020Improving Factual Completeness and Consistency of Image-to-Text Radiology Report Generationcited by 141
  8. 2023Visual Language Pretrained Multiple Instance Zero-Shot Transfer for Histopathology Imagescited by 134
  9. 2022Multi-Modal Masked Autoencoders for Medical Vision-and-Language Pre-Trainingcited by 104
  10. 2021Multiple Instance Captioning: Learning Representations from Histopathology Textbooks and Articlescited by 58
  11. 2023Towards a Visual-Language Foundation Model for Computational Pathologycited by 52
  12. 2021SlideGraph+: Whole Slide Image Level Graphs to Predict HER2Status in Breast Cancercited by 31
  13. 2021HEROHE Challenge: assessing HER2 status in breast cancer without immunohistochemistry or in situ hybridizationcited by 8

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2024: 6; 2025: 10; 2026: 18.

34 papers.

Sort:

By area:

  1. 2024SlideChat: A Large Vision-Language Assistant for Whole-Slide Pathology Image Understandingcited by 57
  2. 2024WSI-LLaVA: A Multimodal Large Language Model for Whole Slide Imagecited by 40
  3. 2024CPath-Omni: A Unified Multimodal Foundation Model for Patch and Whole Slide Image Analysis in Computational Pathologycited by 34
  4. 2025CPathAgent: An Agent-based Foundation Model for Interpretable High-Resolution Pathology Image Analysis Mimicking Pathologists' Diagnostic Logiccited by 29
  5. 2024A New Era in Computational Pathology: A Survey on Foundation and Vision-Language Modelscited by 9
  6. 2025Multi-Modal Foundation Models for Computational Pathology: A Surveycited by 9
  7. 2024Cost-effective Instruction Learning for Pathology Vision and Language Analysiscited by 7
  8. 2025Navigating Gigapixel Pathology Images with Large Multimodal Modelscited by 7
  9. 2026PathNavigate: A Training-Free Pathology Agent with Surprise-Guided Scan and Shared Slide Memory for Whole-Slide Image VQAcited by 7
  10. 2025Efficient Whole Slide Pathology VQA via Token Compressioncited by 6
  11. 2025Content Generation Models in Computational Pathology: A Comprehensive Survey on Methods, Applications, and Challengescited by 5
  12. 2026Act Like a Pathologist: Tissue-Aware Whole Slide Image Reasoningcited by 5
22 more
  1. 2025Beyond Pixel Simulation: Pathology Image Generation via Diagnostic Semantic Tokens and Prototype Controlcited by 4
  2. 2026QCAgent: An agentic framework for quality-controllable pathology report generation from whole slide imagecited by 4
  3. 2025ChatEXAONEPath: An Expert-level Multimodal Large Language Model for Histopathology Using Whole Slide Imagescited by 3
  4. 2026MLLM-HWSI: A Multimodal Large Language Model for Hierarchical Whole Slide Image Understandingcited by 3
  5. 2026Enhancing Pathological VLMs with Cross-scale Reasoningcited by 3
  6. 2026PathSelect: Sequential Token Selection for Whole Slide Pathologycited by 2
  7. 2025The Impact of Image Resolution on Biomedical Multimodal Large Language Modelscited by 1
  8. 2026CTIS-QA: Clinical Template-Informed Slide-level Question Answering for Pathologycited by 1
  9. 2026PathMem: Toward Cognition-Aligned Memory Transformation for Pathology MLLMscited by 1
  10. 2026Towards Clinically Interpretable Ophthalmic VQA via Spatially-Grounded Lesion Evidencecited by 1
  11. 2026Auditing Data Leakage in Whole-Slide Image Multimodal Benchmarkscited by 1
  12. 2026Agentic Visual Reasoning in Whole-Slide Pathology Images via Active Perceptioncited by 1
  13. 2026SlideBank: A Persistent Hierarchical Evidence Bank for Consistent Whole-Slide Reasoningcited by 1
  14. 2024Efficient and Comprehensive Feature Extraction in Large Vision-Language Model for Pathology Analysiscited by 0
  15. 2025VGAT: A Cancer Survival Analysis Framework Transitioning from Generative Visual Question Answering to Genomic Reconstructioncited by 0
  16. 2025PathReasoning: A multimodal reasoning agent for query-based ROI navigation on whole-slide imagescited by 0
  17. 2026Bayesian Integration of Nonlinear Incomplete Clinical Datacited by 0
  18. 2026PBSBench: A Multi-Level Vision-Language Framework and Benchmark for Hematopathology Whole Slide Image Interpretationcited by 0
  19. 2026TaxoMIL: Taxonomy-Constrained Learning for Hierarchical Whole Slide Image Analysiscited by 0
  20. 2026LaGuadia: Language-Guided Adaptive Distillation from Pathology Foundation Modelscited by 0
  21. 2026From Multi-Resolution Cells to Gigapixel Whole Slide Images Foundation Model for Computational Pathologycited by 0
  22. 2026PathoArgus: Advancing Evidence-Grounded Long-Context Visual Reasoning across Gigapixel Whole-Slide and Multi-Slide Case Contextscited by 0