mlsanityData to 7 Oct 2026

A New Era in Computational Pathology: A Survey on Foundation and Vision-Language Models

arXiv 2408.14496 ↗First posted 23 Aug 2024cs.LG cs.AI cs.CLcited by 9

On the map: Multimodal LLMs / vision-language models → Medical vision-language models

Abstract (arXiv)

Recent advances in deep learning have completely transformed the domain of computational pathology (CPath). More specifically, it has altered the diagnostic workflow of pathologists by integrating foundation models (FMs) and vision-language models (VLMs) in their assessment and decision-making process. The limitations of existing deep learning approaches in CPath can be overcome by FMs through learning a representation space that can be adapted to a wide variety of downstream tasks without explicit supervision. Deploying VLMs allow pathology reports written in natural language be used as rich semantic information sources to improve existing models as well as generate predictions in natural language form. In this survey, a holistic and systematic overview of recent innovations in FMs and VLMs in CPath is presented. Furthermore, the tools, datasets and training schemes for these models are summarized in addition to categorizing them into distinct groups. This extensive survey highlights the current trends in CPath and its possible revolution through the use of FMs and VLMs in the future.

Key facts. Cited by 9 papers in the map (data through 7 Oct 2026). Placed in Medical vision-language models, one of 16 directions in Multimodal LLMs / vision-language models. Builds on 133 of its 220 references that are in the map. First posted on arXiv 23 Aug 2024.

Builds on

133 of 220 references.

Closest ancestors · 7

Same direction, newest first

  1. 2024Cost-effective Instruction Learning for Pathology Vision and Language Analysis25 Jul
  2. 2024PathAlign: A vision-language model for whole slide images in histopathology27 Jun
  3. 2024CPLIP: Zero-Shot Learning for Histopathology with Comprehensive Vision-Language Alignment7 Jun
  4. 2024In-context learning enables multimodal large language models to classify cancer pathology images12 Mar
  5. 2024PathMMU: A Massive Multimodal Expert-Level Benchmark for Understanding and Reasoning in Pathology29 Jan
  6. 2024Demonstration of an Adversarial Attack Against a Multimodal Vision Language Model for Pathology Imaging4 Jan
  7. 2023PathAsst: A Generative Foundation AI Assistant Towards Artificial General Intelligence of Pathology24 May

Other references · 126

  1. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,204
  2. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,456
  3. 2020An Image is Worth 16x16 Words: Transformers for Image Recognition at Scalecited by 29,915
  4. 2023GPT-4 Technical Reportcited by 18,132
  5. 2022Training language models to follow instructions with human feedbackcited by 15,712
  6. 2014Very Deep Convolutional Networks for Large-Scale Image Recognitioncited by 15,267
  7. 2024The Llama 3 Herd of Modelscited by 14,192
  8. 2021LoRA: Low-Rank Adaptation of Large Language Modelscited by 14,084
  9. 2023Llama 2: Open Foundation and Fine-Tuned Chat Modelscited by 12,865
  10. 2021Swin Transformer: Hierarchical Vision Transformer using Shifted Windowscited by 12,082
  11. 2014ImageNet Large Scale Visual Recognition Challengecited by 8,441
  12. 2023Visual Instruction Tuningcited by 7,792
38 more
  1. 2023DINOv2: Learning Robust Visual Features without Supervisioncited by 7,764
  2. 2019Momentum Contrast for Unsupervised Visual Representation Learningcited by 7,482
  3. 2021Masked Autoencoders Are Scalable Vision Learnerscited by 7,339
  4. 2021Emerging Properties in Self-Supervised Vision Transformerscited by 6,893
  5. 2016Densely Connected Convolutional Networkscited by 6,691
  6. 2016Grad-CAM: Visual Explanations from Deep Networks via Gradient-based Localizationcited by 6,690
  7. 2017A Unified Approach to Interpreting Model Predictionscited by 6,488
  8. 2019EfficientNet: Rethinking Model Scaling for Convolutional Neural Networkscited by 6,415
  9. 2017Towards Deep Learning Models Resistant to Adversarial Attackscited by 6,326
  10. 2023BLIP-2: Bootstrapping Language-Image Pre-training with Frozen Image Encoders and Large Language Modelscited by 6,069
  11. 2015You Only Look Once: Unified, Real-Time Object Detectioncited by 4,829
  12. 2020Bootstrap your own latent: A new approach to self-supervised Learningcited by 4,712
  13. 2020Training data-efficient image transformers & distillation through attentioncited by 4,560
  14. 2023Improved Baselines with Visual Instruction Tuningcited by 4,362
  15. 2022Flamingo: a Visual Language Model for Few-Shot Learningcited by 4,337
  16. 2022LAION-5B: An open large-scale dataset for training next generation image-text modelscited by 4,131
  17. 2022Robust Speech Recognition via Large-Scale Weak Supervisioncited by 3,972
  18. 2022A ConvNet for the 2020scited by 3,545
  19. 2022Scaling Instruction-Finetuned Language Modelscited by 2,997
  20. 2020Unsupervised Learning of Visual Features by Contrasting Cluster Assignmentscited by 2,845
  21. 2020Improved Baselines with Momentum Contrastive Learningcited by 2,282
  22. 2019HuggingFace's Transformers: State-of-the-art Natural Language Processingcited by 2,201
  23. 2021Barlow Twins: Self-Supervised Learning via Redundancy Reductioncited by 1,818
  24. 2021An Empirical Study of Training Self-Supervised Vision Transformerscited by 1,610
  25. 2015Show, Attend and Tell: Neural Image Caption Generation with Visual Attentioncited by 1,421
  26. 2020WILDS: A Benchmark of in-the-Wild Distribution Shiftscited by 1,366
  27. 2022CoCa: Contrastive Captioners are Image-Text Foundation Modelscited by 1,265
  28. 2023PaLM 2 Technical Reportcited by 1,154
  29. 2021Perceiver: General Perception with Iterative Attentioncited by 1,089
  30. 2020Domain-Specific Language Model Pretraining for Biomedical Natural Language Processingcited by 996
  31. 2019Publicly Available Clinical BERT Embeddingscited by 995
  32. 2021Scaling Vision Transformerscited by 993
  33. 2022YOLOv7: Trainable bag-of-freebies sets new state-of-the-art for real-time object detectorscited by 978
  34. 2021iBOT: Image BERT Pre-Training with Online Tokenizercited by 891
  35. 2014Show and Tell: A Neural Image Caption Generatorcited by 877
  36. 2023Vision-Language Models for Vision Tasks: A Surveycited by 834
  37. 2021PVT v2: Improved Baselines with Pyramid Vision Transformercited by 749
  38. 2022EVA: Exploring the Limits of Masked Visual Representation Learning at Scalecited by 685

50 most cited of 126 All 126 (JSON)

Built on by

Cited by year

Citing papers per year: 2025: 9.

9 papers.

  1. 2025A Survey on Computational Pathology Foundation Models: Datasets, Adaptation Strategies, and Evaluation Taskscited by 15
  2. 2025A Survey of Pathology Foundation Model: Progress and Future Directionscited by 14
  3. 2025SPIDER: A Comprehensive Multi-Organ Supervised Pathology Dataset and Baseline Modelscited by 10
  4. 2025Multi-Modal Foundation Models for Computational Pathology: A Surveycited by 9
  5. 2025Multi-Resolution Pathology-Language Pre-training Model with Text-Guided Visual Representationcited by 6
  6. 2025Content Generation Models in Computational Pathology: A Comprehensive Survey on Methods, Applications, and Challengescited by 5
  7. 2025Beyond Diagnostic Performance: Revealing and Quantifying Ethical Risks in Pathology Foundation Modelscited by 4
  8. 2025Investigating Zero-Shot Diagnostic Pathology in Vision-Language Models with Efficient Prompt Designcited by 2
  9. 2025From Classification to Cross-Modal Understanding: Leveraging Vision-Language Models for Fine-Grained Renal Pathologycited by 1

Similar, not cited

Close in wording, no citation either way

  1. 2025PathVLM-R1: A Reinforcement Learning-Driven Reasoning Model for Pathology Visual-Language Tasks
  2. 2024Benchmarking foundation models as feature extractors for weakly-supervised computational pathology
  3. 2025Comparing Computational Pathology Foundation Models using Representational Similarity Analysis
  4. 2026Benchmarking Vision-Language Models for Automated Pathology Diagnosis and Report Generation
  5. 2025PathoHR: Hierarchical Reasoning for Vision-Language Models in Pathology
  6. 2025A Versatile Pathology Co-pilot via Reasoning Enhanced Multimodal Large Language Model
  7. 2025Effortless Vision-Language Model Specialization in Histopathology without Annotation
  8. 2026FFM-CP: Cross-Backbone Fusion of Vision-Language Foundation Models for Few-Shot Computational Pathology