mlsanityData to 14 Oct 2026

Learning from True-False Labels via Multi-modal Prompt Retrieving

arXiv 2405.15228 ↗First posted 24 May 2024cs.LG cs.CVcited by 1

On the map: Multimodal LLMs / vision-language models → Prompt learning and adaptation of CLIP

Abstract (arXiv)

Pre-trained Vision-Language Models (VLMs) exhibit strong zero-shot classification abilities, demonstrating great potential for generating weakly supervised labels. Unfortunately, existing weakly supervised learning methods are short of ability in generating accurate labels via VLMs. In this paper, we propose a novel weakly supervised labeling setting, namely True-False Labels (TFLs) which can achieve high accuracy when generated by VLMs. The TFL indicates whether an instance belongs to the label, which is randomly and uniformly sampled from the candidate label set. Specifically, we theoretically derive a risk-consistent estimator to explore and utilize the conditional probability distribution information of TFLs. Besides, we propose a convolutional-based Multi-modal Prompt Retrieving (MRP) method to bridge the gap between the knowledge of VLMs and target learning tasks. Experimental results demonstrate the effectiveness of the proposed TFL setting and MRP learning method. The code to reproduce the experiments is at https://github.com/Tranquilxu/TMP.

Key facts. Cited by 1 paper in the map (data through 14 Oct 2026). Placed in Prompt learning and adaptation of CLIP, one of 16 directions in Multimodal LLMs / vision-language models. Builds on 34 of its 60 references that are in the map. First posted on arXiv 24 May 2024.

Builds on

34 of 60 references.

Closest ancestors · 3

Same direction, newest first

  1. 2023Enhancing CLIP with CLIP: Exploring Pseudolabeling for Limited-Label Prompt Tuning2 Jun
  2. 2022Conditional Prompt Learning for Vision-Language Models10 Mar
  3. 2021Learning to Prompt for Vision-Language Models2 Sep

Other references · 31

  1. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,744
  2. 2020An Image is Worth 16x16 Words: Transformers for Image Recognition at Scalecited by 30,186
  3. 2023Visual Instruction Tuningcited by 7,869
  4. 2022BLIP: Bootstrapping Language-Image Pre-training for Unified Vision-Language Understanding and Generationcited by 4,322
  5. 2018Encoder-Decoder with Atrous Separable Convolution for Semantic Image Segmentationcited by 3,223
  6. 2021Pre-train, Prompt, and Predict: A Systematic Survey of Prompting Methods in Natural Language Processingcited by 2,234
  7. 2020FixMatch: Simplifying Semi-Supervised Learning with Consistency and Confidencecited by 2,085
  8. 2022Visual Prompt Tuningcited by 1,681
  9. 2019MixMatch: A Holistic Approach to Semi-Supervised Learningcited by 1,391
  10. 2017Virtual Adversarial Training: A Regularization Method for Supervised and Semi-Supervised Learningcited by 920
  11. 2022MaPLe: Multi-modal Prompt Learningcited by 687
  12. 2017Robust Loss Functions under Label Noise for Deep Neural Networkscited by 443
19 more
  1. 2016Regularization With Stochastic Transformations and Perturbations for Deep Semi-Supervised Learningcited by 441
  2. 2021Pervasive Label Errors in Test Sets Destabilize Machine Learning Benchmarkscited by 395
  3. 2022Exploring Visual Prompts for Adapting Large-Scale Modelscited by 274
  4. 2022DETRs with Collaborative Hybrid Assignments Trainingcited by 274
  5. 2021Open-World Semi-Supervised Learningcited by 165
  6. 2017Learning with Biased Complementary Labelscited by 93
  7. 2020Provably Consistent Partial-Label Learningcited by 92
  8. 2017Learning from Complementary Labelscited by 88
  9. 2022Debiased Learning from Naturally Imbalanced Pseudo-Labelscited by 85
  10. 2020The Implicit Regularization of Stochastic Gradient Flow for Least Squarescited by 59
  11. 2018Complementary-Label Learning for Arbitrary Losses and Modelscited by 57
  12. 2019Learning with Multiple Complementary Labelscited by 53
  13. 2022One Positive Label is Sufficient: Single-Positive Multi-Label Learning with Label Enhancementcited by 22
  14. 2017Semi-Supervised AUC Optimization based on Positive-Unlabeled Learningcited by 20
  15. 2023Towards Effective Visual Representations for Partial-Label Learningcited by 11
  16. 2023MarginMatch: Improving Semi-Supervised Learning with Pseudo-Marginscited by 10
  17. 2023Learning with Complementary Labels Revisited: The Selected-Completely-at-Random Setting Is More Practicalcited by 10
  18. 2022Class-Imbalanced Complementary-Label Learning via Weighted Losscited by 7
  19. 2024Learning from Reduced Labels for Long-Tailed Datacited by 2

Built on by

Cited by year

Citing papers per year: 2025: 1.

1 paper.

  1. 2025Human-Corrected Labels Learning: Enhancing Labels Quality via Human Correction of VLMs Discrepanciescited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2024Tuning Vision-Language Models with Candidate Labels by Prompt Alignment
  2. 2024Candidate Pseudolabel Learning: Enhancing Vision-Language Models by Prompt Tuning with Unlabeled Data
  3. 2023Vision-Language Pseudo-Labels for Single-Positive Multi-Label Learning
  4. 2024Rethinking Prompting Strategies for Multi-Label Recognition with Partial Annotations
  5. 2025SPARC: Score Prompting and Adaptive Fusion for Zero-Shot Multi-Label Recognition in Vision-Language Models
  6. 2025SelfPrompt: Confidence-Aware Semi-Supervised Tuning for Robust Vision-Language Model Adaptation
  7. 2024Data-free Multi-label Image Recognition via LLM-powered Prompt Tuning
  8. 2025Bridging Weakly-Supervised Learning and VLM Distillation: Noisy Partial Label Learning for Efficient Downstream Adaptation