mlsanityData to 7 Oct 2026

Text-guided Foundation Model Adaptation for Pathological Image Classification

arXiv 2307.14901 ↗First posted 27 Jul 2023cs.CV cs.AIcited by 47

Not on the map. Why?

Abstract (arXiv)

The recent surge of foundation models in computer vision and natural language processing opens up perspectives in utilizing multi-modal clinical data to train large models with strong generalizability. Yet pathological image datasets often lack biomedical text annotation and enrichment. Guiding data-efficient image diagnosis from the use of biomedical text knowledge becomes a substantial interest. In this paper, we propose to Connect Image and Text Embeddings (CITE) to enhance pathological image classification. CITE injects text insights gained from language models pre-trained with a broad range of biomedical texts, leading to adapt foundation models towards pathological image understanding. Through extensive experiments on the PatchGastric stomach tumor pathological image dataset, we demonstrate that CITE achieves leading performance compared with various baselines especially when training data is scarce. CITE offers insights into leveraging in-domain text knowledge to reinforce data-efficient pathological image classification. Code is available at https://github.com/Yunkun-Zhang/CITE.

Key facts. Cited by 47 papers in the map (data through 7 Oct 2026). Builds on 20 of its 31 references that are in the map. First posted on arXiv 27 Jul 2023.

Builds on

20 of 31 references.

References · 20

  1. 2015Deep Residual Learning for Image Recognitioncited by 51,809
  2. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,204
  3. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,456
  4. 2020Language Models are Few-Shot Learnerscited by 32,199
  5. 2020An Image is Worth 16x16 Words: Transformers for Image Recognition at Scalecited by 29,915
  6. 2014ImageNet Large Scale Visual Recognition Challengecited by 8,441
  7. 2023BLIP-2: Bootstrapping Language-Image Pre-training with Frozen Image Encoders and Large Language Modelscited by 6,069
  8. 2021On the Opportunities and Risks of Foundation Modelscited by 3,961
  9. 2021Learning to Prompt for Vision-Language Modelscited by 2,269
  10. 2019BioBERT: a pre-trained biomedical language representation model for biomedical text miningcited by 1,935
  11. 2022Visual Prompt Tuningcited by 1,674
  12. 2019Publicly Available Clinical BERT Embeddingscited by 995
8 more
  1. 2020Data Efficient and Weakly Supervised Computational Pathology on Whole Slide Imagescited by 738
  2. 2021LiT: Zero-Shot Transfer with Locked-image text Tuningcited by 572
  3. 2022LinkBERT: Pretraining Language Models with Document Linkscited by 209
  4. 2021VisualGPT: Data-efficient Adaptation of Pretrained Language Models for Image Captioningcited by 190
  5. 2020Meta-Baseline: Exploring Simple Meta-Learning for Few-Shot Learningcited by 179
  6. 2023Learning Customized Visual Models with Retrieval-Augmented Knowledgecited by 58
  7. 2021INTERN: A New Learning Paradigm Towards General Visioncited by 33
  8. 2022Inference of captions from histopathological patchescited by 26

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2023: 6; 2024: 23; 2025: 15; 2026: 3.

47 papers.

Sort:

By area:

  1. 2024A Comprehensive Survey of Scientific Large Language Models and Their Applications in Scientific Discoverycited by 96
  2. 2024Foundation Model for Advancing Healthcare: Challenges, Opportunities, and Future Directionscited by 77
  3. 2023Foundational Models in Medical Imaging: A Comprehensive Survey and Future Visioncited by 74
  4. 2024A Comprehensive Survey of Foundation Models in Medicinecited by 56
  5. 2023CLIP in Medical Imaging: A Surveycited by 40
  6. 2024PathAlign: A vision-language model for whole slide images in histopathologycited by 29
  7. 2025ViLa-MIL: Dual-scale Vision-Language Multiple Instance Learning for Whole Slide Image Classificationcited by 28
  8. 2024Data-Centric Foundation Models in Computational Healthcare: A Surveycited by 27
  9. 2024Generalizable Whole Slide Image Classification with Fine-Grained Visual-Semantic Interactioncited by 27
  10. 2023Medical Vision Language Pretraining: A surveycited by 24
  11. 2024Benchmarking Pathology Foundation Models: Adaptation Strategies and Scenarioscited by 16
  12. 2024BAPLe: Backdoor Attacks on Medical Foundational Models using Prompt Learningcited by 15
35 more
  1. 2023PathoDuet: Foundation Models for Pathological Slide Analysis of H&E and IHC Stainscited by 14
  2. 2024PathM3: A Multimodal Multi-Task Multiple Instance Learning Framework for Whole Slide Image Classification and Captioningcited by 13
  3. 2024Pathology-knowledge Enhanced Multi-instance Prompt Learning for Few-shot Whole Slide Image Classificationcited by 13
  4. 2024Free Lunch in Pathology Foundation Model: Task-specific Model Adaptation with Concept-Guided Feature Enhancementcited by 11
  5. 2024DeViDe: Faceted medical knowledge for improved medical vision-language pre-trainingcited by 10
  6. 2024A New Era in Computational Pathology: A Survey on Foundation and Vision-Language Modelscited by 9
  7. 2025Taming Vision-Language Models for Medical Image Analysis: A Comprehensive Reviewcited by 9
  8. 2024PathoTune: Adapting Visual Foundation Model to Pathological Specialistscited by 8
  9. 2024FAST: A Dual-tier Few-Shot Learning Paradigm for Whole Slide Image Classificationcited by 8
  10. 2024Embedded Visual Prompt Tuningcited by 7
  11. 2024Cost-effective Instruction Learning for Pathology Vision and Language Analysiscited by 7
  12. 2025Foundation Models in Medical Imaging: A Review and Outlookcited by 7
  13. 2024Clinical-grade Multi-Organ Pathology Report Generation for Multi-scale Whole Slide Images via a Semantically Guided Medical Text Foundation Modelcited by 6
  14. 2026Multimodal Model for Computational Pathology:Representation Learning and Image Compressioncited by 6
  15. 2024OpenMEDLab: An Open-source Platform for Multi-modality Foundation Models in Medicinecited by 5
  16. 2024Advancing COVID-19 Detection in 3D CT Scanscited by 5
  17. 2025CognoSpeak: an automatic, remote assessment of early cognitive decline in real-world conversational speechcited by 5
  18. 2024Diagnostic Text-guided Representation Learning in Hierarchical Classification for Pathological Whole Slide Imagecited by 4
  19. 2025FedEFM: Federated Endovascular Foundation Model with Unseen Datacited by 2
  20. 2025Conformal uncertainty quantification to evaluate predictive fairness of foundation AI model for skin lesion classes across patient demographicscited by 2
  21. 2025HiLa: Hierarchical Vision-Language Collaboration for Cancer Survival Predictioncited by 2
  22. 2023Zero-Shot Refinement of Buildings' Segmentation Models using SAMcited by 1
  23. 2024Benchmarking Vision-Language Contrastive Methods for Medical Representation Learningcited by 1
  24. 2024Foundational Models for Pathology and Endoscopy Images: Application for Gastric Inflammationcited by 1
  25. 2025MedForge: Building Medical Foundation Models Like Open Source Software Developmentcited by 1
  26. 2025A Shared Encoder Approach to Multimodal Representation Learningcited by 1
  27. 2025Slide-Level Prompt Learning with Vision Language Models for Few-Shot Multiple Instance Learning in Histopathologycited by 1
  28. 2025MedSapiens: Taking a Pose to Rethink Medical Imaging Landmark Detectioncited by 1
  29. 2023TransMed: Large Language Models Enhance Vision Transformer for Biomedical Image Classificationcited by 0
  30. 2025Robust sensitivity control in digital pathology via tile score distribution matchingcited by 0
  31. 2025Prompt Mechanisms in Medical Imaging: A Comprehensive Surveycited by 0
  32. 2025Enriched text-guided variational multimodal knowledge distillation network (VMD) for automated diagnosis of plaque vulnerability in 3D carotid artery MRIcited by 0
  33. 2025Leveraging Text Guidance for Enhancing Demographic Fairness in Gender Classificationcited by 0
  34. 2026Hierarchical Prototype-based Domain Priors for Multiple Instance Learning in Multimodal Histopathology Analysiscited by 0
  35. 2026Modeling Whole-Slide Images as Dynamic Tumor Microenvironment Fieldscited by 0