mlsanityData to 7 Oct 2026

PRISM2: Unlocking Multi-Modal General Pathology AI with Clinical Dialogue

arXiv 2506.13063 ↗First posted 16 Jun 2025cs.CV cs.CLcited by 8

Not on the map. Why?

Abstract (arXiv)

Recent rapid progress in the field of computational pathology has been enabled by foundation models. These models are beginning to move beyond encoding image patches towards whole-slide understanding but their clinical utility remains limited. In this work, we present PRISM2, a multimodal slide-level foundation model trained on data from 700,000 diagnostic specimen-report pairs, the largest vision (2.3 million whole slide images) and language (14M question-answer pairs) histopathology dataset to date. By learning through clinical-dialogue supervision, PRISM2 aligns histomorphologic features with the language of diagnostic reasoning, producing slide-level representations that support both direct diagnostic question-answering and transferable embeddings for downstream tasks. Without additional training, PRISM2 matches or exceeds the cancer-detection performance of clinical-grade products. This is observed without loss of generality on other tasks, where PRISM2 achieves top performance. Finally, using survival prediction as the example, we show that task-specific finetuning with a large dataset can outperform task-specific models, further improving performance. These results demonstrate how language-supervised pretraining provides a scalable, clinically grounded signal for learning generalizable pathology representations, bridging human diagnostic reasoning and foundation-model performance.

Key facts. Cited by 8 papers in the map (data through 7 Oct 2026). Builds on 27 of its 54 references that are in the map. First posted on arXiv 16 Jun 2025.

Builds on

27 of 54 references.

References · 27

  1. 2023Visual Instruction Tuningcited by 7,792
  2. 2023DINOv2: Learning Robust Visual Features without Supervisioncited by 7,764
  3. 2020Scaling Laws for Neural Language Modelscited by 6,411
  4. 2024GPT-4o System Cardcited by 4,467
  5. 2022FlashAttention: Fast and Memory-Efficient Exact Attention with IO-Awarenesscited by 3,611
  6. 2023FlashAttention-2: Faster Attention with Better Parallelism and Work Partitioningcited by 2,551
  7. 2016Making the V in VQA Matter: Elevating the Role of Image Understanding in Visual Question Answeringcited by 2,498
  8. 2024Phi-3 Technical Report: A Highly Capable Language Model Locally on Your Phonecited by 1,883
  9. 2023GQA: Training Generalized Multi-Query Transformer Models from Multi-Head Checkpointscited by 1,311
  10. 2022CoCa: Contrastive Captioners are Image-Text Foundation Modelscited by 1,265
  11. 2021Perceiver: General Perception with Iterative Attentioncited by 1,089
  12. 2021Scaling Vision Transformerscited by 993
15 more
  1. 2022BioGPT: Generative Pre-trained Transformer for Biomedical Text Generation and Miningcited by 531
  2. 2022Scaling Vision Transformers to Gigapixel Images via Hierarchical Self-Supervised Learningcited by 357
  3. 2024Multimodal Whole Slide Foundation Model for Pathologycited by 156
  4. 2024Virchow2: Scaling Self-Supervised Mixed Magnification Models in Pathologycited by 144
  5. 2024MAIRA-2: Grounded Radiology Report Generationcited by 140
  6. 2023Artificial Intelligence for Digital and Computational Pathologycited by 106
  7. 2022Head2Toe: Utilizing Intermediate Representations for Better Transfer Learningcited by 83
  8. 2024PRISM: A Multi-Modal Generative Foundation Model for Slide-Level Histopathologycited by 80
  9. 2024A Multimodal Knowledge-enhanced Whole-slide Pathology Foundation Modelcited by 59
  10. 2025Molecular-driven Foundation Model for Oncologic Pathologycited by 52
  11. 2024Transcriptomics-guided Slide Representation Learning in Computational Pathologycited by 50
  12. 2024PathAlign: A vision-language model for whole slide images in histopathologycited by 29
  13. 2024Multistain Pretraining for Slide Representation Learning in Pathologycited by 27
  14. 2024Unsupervised Foundation Model-Agnostic Slide-Level Representation Learningcited by 15
  15. 2024Screen Them All: High-Throughput Pan-Cancer Genetic and Phenotypic Biomarker Screening from H&E Whole Slide Imagescited by 8

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2025: 5; 2026: 3.

8 papers.