mlsanityData to 14 Oct 2026

Generalizable Prompt Learning of CLIP: A Brief Overview

arXiv 2503.01263 ↗First posted 3 Mar 2025cs.CV cs.CLcited by 3

On the map: Multimodal LLMs / vision-language models → Prompt learning and adaptation of CLIP

Abstract (arXiv)

Existing vision-language models (VLMs) such as CLIP have showcased an impressive capability to generalize well across various downstream tasks. These models leverage the synergy between visual and textual information, enabling them to understand and reason about the content present in images and text in a unified manner. This article provides a brief overview of CLIP based on few-shot prompt learning, including experimental data and technical characteristics of some methods. The purpose of this review is to provide a reference for researchers who have just started their research in generalizable prompting of CLIP through few-shot training for classification across 15 datasets and also to facilitate the integration of this field by researchers in other downstream tasks.

Key facts. Cited by 3 papers in the map (data through 14 Oct 2026). Placed in Prompt learning and adaptation of CLIP, one of 16 directions in Multimodal LLMs / vision-language models. Builds on 106 of its 148 references that are in the map. First posted on arXiv 3 Mar 2025.

Builds on

106 of 148 references.

Closest ancestors · 21

Same direction, newest first

  1. 2025A Similarity Paradigm Through Textual Regularization Without Forgetting20 Feb
  2. 2024Advancing Prompt Learning through an External Layer29 Jul
  3. 2024Visual-Text Cross Alignment: Refining the Similarity Score in Vision-Language Models5 Jun
  4. 2024AAPL: Adding Attributes to Prompt Learning for Vision-Language Models25 Apr
  5. 2024Prompt Learning via Meta-Regularization1 Apr
  6. 2024Negative Label Guided OOD Detection with Pretrained Vision-Language Models29 Mar
  7. 2023Domain Prompt Learning with Quaternion Networks12 Dec
  8. 2023TCP:Textual-based Class-aware Prompt tuning for Visual-Language Model30 Nov
  9. 2023Beyond Sole Strength: Customized Ensembles for Generalized Vision-Language Models28 Nov
  10. 2023HGCLIP: Exploring Vision-Language Models with Graph Representations for Hierarchical Understanding23 Nov
  11. 2023Domain-Controlled Prompt Learning30 Sep
  12. 2023Tuning Multi-mode Token-level Prompt Alignment across Modalities25 Sep
9 more
  1. 2023CoPL: Contextual Prompt Learning for Vision-Language Understanding3 Jul
  2. 2023Consistency-guided Prompt Learning for Vision-Language Models1 Jun
  3. 2023Visual-Language Prompt Tuning with Knowledge-guided Context Optimization23 Mar
  4. 2022Test-Time Prompt Tuning for Zero-Shot Generalization in Vision-Language Models15 Sep
  5. 2022Prompt-aligned Gradient for Prompt Tuning30 May
  6. 2022Prompt Distribution Learning6 May
  7. 2022Unsupervised Prompt Learning for Vision-Language Models7 Apr
  8. 2022Conditional Prompt Learning for Vision-Language Models10 Mar
  9. 2021Learning to Prompt for Vision-Language Models2 Sep

Other references · 85

  1. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,399
  2. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,744
  3. 2020An Image is Worth 16x16 Words: Transformers for Image Recognition at Scalecited by 30,186
  4. 2019Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformercited by 14,599
  5. 2017Towards Deep Learning Models Resistant to Adversarial Attackscited by 6,415
  6. 2021Prefix-Tuning: Optimizing Continuous Prompts for Generationcited by 3,915
  7. 2021Scaling Up Visual and Vision-Language Representation Learning With Noisy Text Supervisioncited by 3,864
  8. 2021The Power of Scale for Parameter-Efficient Prompt Tuningcited by 3,795
  9. 2013Sinkhorn Distances: Lightspeed Computation of Optimal Transportation Distancescited by 2,316
  10. 2013Describing Textures in the Wildcited by 2,125
  11. 2020The Many Faces of Robustness: A Critical Analysis of Out-of-Distribution Generalizationcited by 1,850
  12. 2013Fine-Grained Visual Classification of Aircraftcited by 1,726
38 more
  1. 2022Visual Prompt Tuningcited by 1,681
  2. 2017EuroSAT: A Novel Dataset and Deep Learning Benchmark for Land Use and Land Cover Classificationcited by 1,496
  3. 2019Natural Adversarial Examplescited by 1,408
  4. 2019Learning Robust Global Representations by Penalizing Local Predictive Powercited by 1,013
  5. 2017Learning to Generalize: Meta-Learning for Domain Generalizationcited by 763
  6. 2020Out-of-Distribution Generalization via Risk Extrapolation (REx)cited by 742
  7. 2022Fine-Tuning can Distort Pretrained Features and Underperform Out-of-Distributioncited by 739
  8. 2022EVA: Exploring the Limits of Masked Visual Representation Learning at Scalecited by 691
  9. 2021P-Tuning v2: Prompt Tuning Can Be Comparable to Fine-tuning Universally Across Scales and Taskscited by 689
  10. 2022MaPLe: Multi-modal Prompt Learningcited by 687
  11. 2013Domain Generalization via Invariant Feature Representationcited by 623
  12. 2019ActivityNet-QA: A Dataset for Understanding Complex Web Videos via Question Answeringcited by 561
  13. 2018Generalizing to Unseen Domains via Adversarial Data Augmentationcited by 451
  14. 2019Domain Generalization by Solving Jigsaw Puzzlescited by 450
  15. 2021SWAD: Domain Generalization by Seeking Flat Minimacited by 434
  16. 2019Deep Modular Co-Attention Networks for Visual Question Answeringcited by 339
  17. 2023Self-regulating Prompts: Foundational Model Adaptation without Forgettingcited by 272
  18. 2017Zero-Shot Learning -- The Good, the Bad and the Uglycited by 255
  19. 2021Combined Scaling for Zero-shot Transfer Learningcited by 182
  20. 2022SegCLIP: Patch Aggregation with Learnable Centers for Open-Vocabulary Semantic Segmentationcited by 164
  21. 2022Finetune like you pretrain: Improved finetuning of zero-shot vision modelscited by 150
  22. 2022Diverse Weight Averaging for Out-of-Distribution Generalizationcited by 148
  23. 2021Ensemble of Averages: Improving Model Selection and Boosting Performance in Domain Generalizationcited by 144
  24. 2022Learning Causally Invariant Representations for Out-of-Distribution Generalization on Graphscited by 144
  25. 2020Efficient Domain Generalization via Common-Specific Low-Rank Decompositioncited by 134
  26. 2022Open-Vocabulary Universal Image Segmentation with MaskCLIPcited by 132
  27. 2019Feature-Critic Networks for Heterogeneous Domain Generalizationcited by 130
  28. 2020Learning to Balance Specificity and Invariance for In and Out of Domain Generalizationcited by 128
  29. 2017Multi-modal Factorized Bilinear Pooling with Co-Attention Learning for Visual Question Answeringcited by 117
  30. 2022Is Out-of-Distribution Detection Learnable?cited by 109
  31. 2022Model Ratatouille: Recycling Diverse Models for Out-of-Distribution Generalizationcited by 97
  32. 2023CLIPood: Generalizing CLIP to Out-of-Distributionscited by 96
  33. 2017Beyond Bilinear: Generalized Multimodal Factorized High-order Pooling for Visual Question Answeringcited by 79
  34. 2018Optimal Transport for Multi-source Domain Adaptation under Target Shiftcited by 68
  35. 2023DePT: Decoupled Prompt Tuningcited by 58
  36. 2019Open Set Domain Adaptation: Theoretical Bound and Algorithmcited by 48
  37. 2024One Prompt Word is Enough to Boost Adversarial Robustness for Pre-trained Vision-Language Modelscited by 48
  38. 2024Few-Shot Adversarial Prompt Learning on Vision-Language Modelscited by 39

50 most cited of 85 All 85 (JSON)

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2025: 1; 2026: 2.

3 papers.

  1. 2025Enhancing Target-unspecific Tasks through a Features Matrixcited by 2
  2. 2026Let's Roll a BiFTA: Bi-refinement for Fine-grained Text-visual Alignment in Vision-Language Modelscited by 1
  3. 2026Joint Semantic Token Selection and Prompt Optimization for Interpretable Prompt Learningcited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2024Fully Fine-tuned CLIP Models are Efficient Few-Shot Learners
  2. 2024Learning to Prompt with Text Only Supervision for Vision-Language Models
  3. 2023Improving Zero-Shot Generalization for CLIP with Synthesized Prompts
  4. 2024Aggregate-and-Adapt Natural Language Prompts for Downstream Generalization of CLIP
  5. 2025Integrated Structural Prompt Learning for Vision-Language Models
  6. 2024Generalizable Prompt Tuning for Vision-Language Models
  7. 2023Enhancing CLIP with GPT-4: Harnessing Visual Descriptions as Prompts
  8. 2023SYNC-CLIP: Synthetic Data Make CLIP Generalize Better in Data-Limited Scenarios