mlsanityData to 7 Oct 2026

Candidate Pseudolabel Learning: Enhancing Vision-Language Models by Prompt Tuning with Unlabeled Data

arXiv 2406.10502 ↗First posted 15 Jun 2024cs.LG cs.AI cs.CVcited by 19

On the map: Multimodal LLMs / vision-language models → Prompt learning and adaptation of CLIP

Abstract (arXiv)

Fine-tuning vision-language models (VLMs) with abundant unlabeled data recently has attracted increasing attention. Existing methods that resort to the pseudolabeling strategy would suffer from heavily incorrect hard pseudolabels when VLMs exhibit low zero-shot performance in downstream tasks. To alleviate this issue, we propose a Candidate Pseudolabel Learning method, termed CPL, to fine-tune VLMs with suitable candidate pseudolabels of unlabeled data in downstream tasks. The core of our method lies in the generation strategy of candidate pseudolabels, which progressively generates refined candidate pseudolabels by both intra- and inter-instance label selection, based on a confidence score matrix for all unlabeled data. This strategy can result in better performance in true label inclusion and class-balanced instance selection. In this way, we can directly apply existing loss functions to learn with generated candidate psueudolabels. Extensive experiments on nine benchmark datasets with three learning paradigms demonstrate the effectiveness of our method. Our code can be found at https://github.com/vanillaer/CPL-ICML2024.

Key facts. Cited by 19 papers in the map (data through 7 Oct 2026). Placed in Prompt learning and adaptation of CLIP, one of 16 directions in Multimodal LLMs / vision-language models. Builds on 30 of its 45 references that are in the map. First posted on arXiv 15 Jun 2024.

Builds on

30 of 45 references.

Closest ancestors · 6

Same direction, newest first

  1. 2023Enhancing CLIP with CLIP: Exploring Pseudolabeling for Limited-Label Prompt Tuning2 Jun
  2. 2022Test-Time Prompt Tuning for Zero-Shot Generalization in Vision-Language Models15 Sep
  3. 2022Unsupervised Prompt Learning for Vision-Language Models7 Apr
  4. 2022Conditional Prompt Learning for Vision-Language Models10 Mar
  5. 2021Tip-Adapter: Training-free CLIP-Adapter for Better Vision-Language Modeling6 Nov
  6. 2021Learning to Prompt for Vision-Language Models2 Sep

Other references · 24

  1. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,456
  2. 2022Flamingo: a Visual Language Model for Few-Shot Learningcited by 4,337
  3. 2022BLIP: Bootstrapping Language-Image Pre-training for Unified Vision-Language Understanding and Generationcited by 4,310
  4. 2013Describing Textures in the Wildcited by 2,110
  5. 2020FixMatch: Simplifying Semi-Supervised Learning with Consistency and Confidencecited by 2,080
  6. 2013Fine-Grained Visual Classification of Aircraftcited by 1,701
  7. 2022Visual Prompt Tuningcited by 1,674
  8. 2021Align before Fuse: Vision and Language Representation Learning with Momentum Distillationcited by 1,612
  9. 2017EuroSAT: A Novel Dataset and Deep Learning Benchmark for Land Use and Land Cover Classificationcited by 1,488
  10. 2016Unsupervised Learning of Visual Representations by Solving Jigsaw Puzzlescited by 1,379
  11. 2023Vision-Language Models for Vision Tasks: A Surveycited by 834
  12. 2021Florence: A New Foundation Model for Computer Visioncited by 802
12 more
  1. 2017Remote Sensing Image Scene Classification: Benchmark and State of the Artcited by 791
  2. 2021FlexMatch: Boosting Semi-Supervised Learning with Curriculum Pseudo Labelingcited by 537
  3. 2019BBN: Bilateral-Branch Network with Cumulative Learning for Long-Tailed Visual Recognitioncited by 403
  4. 2022Exploring Visual Prompts for Adapting Large-Scale Modelscited by 271
  5. 2021Dash: Semi-Supervised Learning with Dynamic Thresholdingcited by 147
  6. 2022Domain Adaptation via Prompt Learningcited by 128
  7. 2020Provably Consistent Partial-Label Learningcited by 92
  8. 2021Leveraged Weighted Loss for Partial Label Learningcited by 66
  9. 2020A survey on Semi-, Self- and Unsupervised Learning for Image Classificationcited by 55
  10. 2023LaFTer: Label-Free Tuning of Zero-shot Classifier using Language and Unlabeled Image Collectionscited by 43
  11. 2019Transductive Zero-Shot Learning with Visual Structure Constraintcited by 36
  12. 2022Improving Zero-Shot Models with Label Distribution Priorscited by 14

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2024: 2; 2025: 14; 2026: 3.

19 papers.

Sort:

By area:

  1. 2025CLIP-Powered Domain Generalization and Domain Adaptation: A Comprehensive Surveycited by 15
  2. 2024Defending Multimodal Backdoored Models by Repulsive Visual Prompt Tuningcited by 12
  3. 2024Tuning Vision-Language Models with Candidate Labels by Prompt Alignmentcited by 6
  4. 2025Revisiting semi-supervised learning in the era of foundation modelscited by 6
  5. 2025Unlabeled Data vs. Pre-trained Knowledge: Rethinking SSL in the Era of Large Modelscited by 3
  6. 2025FATE: A Prompt-Tuning-Based Semi-Supervised Learning Framework for Extremely Limited Labeled Datacited by 2
  7. 2025Tuning the Right Foundation Models is What you Need for Partial Label Learningcited by 2
  8. 2025Enhancing Semi-supervised Learning with Zero-shot Pseudolabelscited by 1
  9. 2025When Test-Time Adaptation Meets Self-Supervised Modelscited by 1
  10. 2025Towards Fine-Grained Adaptation of CLIP via a Self-Trained Alignment Scorecited by 1
  11. 2025SelfPrompt: Confidence-Aware Semi-Supervised Tuning for Robust Vision-Language Model Adaptationcited by 0
  12. 2025Beyond Words: Augmenting Discriminative Richness via Diffusions in Unsupervised Prompt Learningcited by 0
7 more
  1. 2025Handling Imbalanced Pseudolabels for Vision-Language Models with Concept Alignment and Confusion-Aware Calibrated Margincited by 0
  2. 2025Cooperative Pseudo Labeling for Unsupervised Federated Classificationcited by 0
  3. 2025Bi-CoG: Bi-Consistency-Guided Self-Training for Vision-Language Modelscited by 0
  4. 2025Solving Semi-Supervised Few-Shot Learning from an Auto-Annotation Perspectivecited by 0
  5. 2026Closing the Confusion Loop: CLIP-Guided Alignment for Source-Free Domain Adaptationcited by 0
  6. 2026ConfusionBench: An Expert-Validated Benchmark for Confusion Recognition and Localization in Educational Videoscited by 0
  7. 2026SECOS: Semantic Capture for Rigorous Classification in Open-World Semi-Supervised Learningcited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2025Vision-Language Model Selection and Reuse for Downstream Adaptation
  2. 2023Vision-Language Pseudo-Labels for Single-Positive Multi-Label Learning
  3. 2023Q: How to Specialize Large Vision-Language Models to Data-Scarce VQA Tasks? A: Self-Train on Unlabeled Images!
  4. 2026Fine-tuning Pre-trained Vision-Language Models in a Human-Annotation-Free Manner
  5. 2024Efficient and Context-Aware Label Propagation for Zero-/Few-Shot Training-Free Adaptation of Vision-Language Model
  6. 2025Adapting Vision-Language Models Without Labels: A Comprehensive Survey
  7. 2026From Inference to Adaptation: A Unified Optimal Transport View of Vision Language Model
  8. 2024Learning from True-False Labels via Multi-modal Prompt Retrieving