mlsanityData to 14 Oct 2026

Semi-Supervised Few-Shot Adaptation of Vision-Language Models

arXiv 2603.02959 ↗First posted 3 Mar 2026cs.CVcited by 0

On the map: Multimodal LLMs / vision-language models → Prompt learning and adaptation of CLIP

Abstract (arXiv)

Vision-language models (VLMs) pre-trained on large, heterogeneous data sources are becoming increasingly popular, providing rich multi-modal embeddings that enable efficient transfer to new tasks. A particularly relevant application is few-shot adaptation, where only a handful of annotated examples are available to adapt the model through multi-modal linear probes. In medical imaging, specialized VLMs have shown promising performance in zero- and few-shot image classification, which is valuable for mitigating the high cost of expert annotations. However, challenges remain in extremely low-shot regimes: the inherent class imbalances in medical tasks often lead to underrepresented categories, penalizing overall model performance. To address this limitation, we propose leveraging unlabeled data by introducing an efficient semi-supervised solver that propagates text-informed pseudo-labels during few-shot adaptation. The proposed method enables lower-budget annotation pipelines for adapting VLMs, reducing labeling effort by >50% in low-shot regimes.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Placed in Prompt learning and adaptation of CLIP, one of 16 directions in Multimodal LLMs / vision-language models. Builds on 28 of its 43 references that are in the map. First posted on arXiv 3 Mar 2026.

Builds on

28 of 43 references.

Closest ancestors · 5

Same direction, newest first

  1. 2024Boosting Vision-Language Models with Transduction3 Jun
  2. 2023A Closer Look at the Few-Shot Adaptation of Large Vision-Language Models20 Dec
  3. 2022Task Residual for Tuning Vision-Language Models18 Nov
  4. 2022Test-Time Prompt Tuning for Zero-Shot Generalization in Vision-Language Models15 Sep
  5. 2021Learning to Prompt for Vision-Language Models2 Sep

Other references · 23

  1. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,744
  2. 2013Sinkhorn Distances: Lightspeed Computation of Optimal Transportation Distancescited by 2,316
  3. 2020FixMatch: Simplifying Semi-Supervised Learning with Consistency and Confidencecited by 2,085
  4. 2019CheXpert: A Large Chest Radiograph Dataset with Uncertainty Labels and Expert Comparisoncited by 1,423
  5. 2017ChestX-ray8: Hospital-scale Chest X-ray Database and Benchmarks on Weakly-Supervised Classification and Localization of Common Thorax Diseasescited by 972
  6. 2020Contrastive Learning of Medical Visual Representations from Paired Images and Textcited by 641
  7. 2019PadChest: A large chest x-ray image dataset with multi-label annotated reportscited by 309
  8. 2015Coordinate Descent Algorithmscited by 219
  9. 2020Can AI help in screening Viral and COVID-19 pneumonia?cited by 200
  10. 2019SimpleShot: Revisiting Nearest-Neighbor Classification for Few-Shot Learningcited by 195
  11. 2022Metrics reloaded: Recommendations for image analysis validationcited by 195
  12. 2023Quilt-1M: One Million Image-Text Pairs for Histopathologycited by 194
11 more
  1. 2020A unifying mutual information view of metric learning: cross-entropy vs. pairwise lossescited by 100
  2. 2023Multimodality Helps Unimodality: Cross-Modal Few-Shot Learning with Multimodal Modelscited by 91
  3. 2023A Foundation Language-Image Model of the Retina (FLAIR): Encoding Expert Knowledge in Text Supervisioncited by 78
  4. 2024No "Zero-Shot" Without Exponential Data: Pretraining Concept Frequency Determines Multimodal Model Performancecited by 75
  5. 2021Going Deeper through the Gleason Scoring Scale: An Automatic end-to-end System for Histology Prostate Grading and Cribriform Pattern Detectioncited by 68
  6. 2024LP++: A Surprisingly Strong Linear Probe for Few-Shot CLIPcited by 60
  7. 2022Long-Tailed Classification of Thorax Diseases on Chest X-Ray: A New Benchmark Studycited by 45
  8. 2024Few-shot Adaptation of Medical Vision-Language Modelscited by 24
  9. 2025Full Conformal Adaptation of Medical Vision-Language Modelscited by 9
  10. 2025Few-Shot, Now for Real: Medical VLMs Adaptation without Balanced Sets or Validationcited by 7
  11. 2025A Reality Check of Vision-Language Pre-training in Radiology: Have We Progressed Using Text?cited by 2

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2026TCLA: Training-Free Class-wise Logit Adaptation for Medical Vision-Language Models
  2. 2025Taming Vision-Language Models for Medical Image Analysis: A Comprehensive Review
  3. 2024Low-Rank Few-Shot Adaptation of Vision-Language Models
  4. 2026From Inference to Adaptation: A Unified Optimal Transport View of Vision Language Model
  5. 2025Adapting Vision-Language Models Without Labels: A Comprehensive Survey
  6. 2024Making Large Vision Language Models to be Good Few-shot Learners
  7. 2022SgVA-CLIP: Semantic-guided Visual Adapting of Vision-Language Models for Few-shot Image Classification
  8. 2025Solving Semi-Supervised Few-Shot Learning from an Auto-Annotation Perspective