mlsanityData to 7 Oct 2026

TCP:Textual-based Class-aware Prompt tuning for Visual-Language Model

arXiv 2311.18231 ↗First posted 30 Nov 2023cs.CVcited by 75

On the map: Multimodal LLMs / vision-language models → Prompt learning and adaptation of CLIP

Abstract (arXiv)

Prompt tuning represents a valuable technique for adapting pre-trained visual-language models (VLM) to various downstream tasks. Recent advancements in CoOp-based methods propose a set of learnable domain-shared or image-conditional textual tokens to facilitate the generation of task-specific textual classifiers. However, those textual tokens have a limited generalization ability regarding unseen domains, as they cannot dynamically adjust to the distribution of testing classes. To tackle this issue, we present a novel Textual-based Class-aware Prompt tuning(TCP) that explicitly incorporates prior knowledge about classes to enhance their discriminability. The critical concept of TCP involves leveraging Textual Knowledge Embedding (TKE) to map the high generalizability of class-level textual knowledge into class-aware textual tokens. By seamlessly integrating these class-aware prompts into the Text Encoder, a dynamic class-aware classifier is generated to enhance discriminability for unseen domains. During inference, TKE dynamically generates class-aware prompts related to the unseen classes. Comprehensive evaluations demonstrate that TKE serves as a plug-and-play module effortlessly combinable with existing methods. Furthermore, TCP consistently achieves superior performance while demanding less training time. Code:https://github.com/htyao89/Textual-based_Class-aware_prompt_tuning/

Key facts. Cited by 75 papers in the map (data through 7 Oct 2026). Placed in Prompt learning and adaptation of CLIP, one of 16 directions in Multimodal LLMs / vision-language models. Builds on 20 of its 52 references that are in the map. First posted on arXiv 30 Nov 2023.

Builds on

20 of 52 references.

Closest ancestors · 6

Same direction, newest first

  1. 2023Prompt Tuning based Adapter for Vision-Language Model Adaption24 Mar
  2. 2023Gradient-Regulated Meta-Prompt Learning for Generalizable Vision-Language Models12 Mar
  3. 2022Multitask Vision-Language Prompt Tuning21 Nov
  4. 2022Unified Vision and Language Prompt Learning13 Oct
  5. 2022PLOT: Prompt Learning with Optimal Transport for Vision-Language Models3 Oct
  6. 2021CLIP-Adapter: Better Vision-Language Models with Feature Adapters9 Oct

Other references · 14

  1. 2017Attention Is All You Needcited by 59,932
  2. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,456
  3. 2020An Image is Worth 16x16 Words: Transformers for Image Recognition at Scalecited by 29,915
  4. 2021LoRA: Low-Rank Adaptation of Large Language Modelscited by 14,084
  5. 2022Flamingo: a Visual Language Model for Few-Shot Learningcited by 4,337
  6. 2022BLIP: Bootstrapping Language-Image Pre-training for Unified Vision-Language Understanding and Generationcited by 4,310
  7. 2022LAION-5B: An open large-scale dataset for training next generation image-text modelscited by 4,131
  8. 2021Scaling Up Visual and Vision-Language Representation Learning With Noisy Text Supervisioncited by 3,848
  9. 2019ViLBERT: Pretraining Task-Agnostic Visiolinguistic Representations for Vision-and-Language Taskscited by 2,219
  10. 2013Fine-Grained Visual Classification of Aircraftcited by 1,701
  11. 2021LAION-400M: Open Dataset of CLIP-Filtered 400 Million Image-Text Pairscited by 1,544
  12. 2021ViLT: Vision-and-Language Transformer Without Convolution or Region Supervisioncited by 1,224
2 more
  1. 2021Towards a Unified View of Parameter-Efficient Transfer Learningcited by 863
  2. 2023DePT: Decoupled Prompt Tuningcited by 58

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2024: 10; 2025: 38; 2026: 27.

75 papers.

Sort:

By area:

  1. 2025Parameter-Efficient Fine-Tuning for Foundation Modelscited by 43
  2. 2025MAP-VLA: Memory-Augmented Prompting for Vision-Language-Action Model in Robotic Manipulationcited by 28
  3. 2024AWT: Transferring Vision-Language Models via Augmentation, Weighting, and Transportationcited by 27
  4. 2025DPC: Dual-Prompt Collaboration for Tuning Vision-Language Modelscited by 17
  5. 2025OpenworldAUC: Towards Unified Evaluation and Optimization for Open-world Prompt Tuningcited by 12
  6. 2025Hierarchical Cross-modal Prompt Learning for Vision-Language Modelscited by 12
  7. 2024Skip Tuning: Pre-trained Vision-Language Models are Effective and Efficient Adapters Themselvescited by 10
  8. 2025Mettle: Meta-Token Learning for Memory-Efficient Audio-Visual Adaptationcited by 10
  9. 2024Understanding and Mitigating Miscalibration in Prompt Tuning for Vision-Language Modelscited by 9
  10. 2024DA-Ada: Learning Domain-Aware Adapter for Domain Adaptive Object Detectioncited by 8
  11. 2025Black Sheep in the Herd: Playing with Spuriously Correlated Attributes for Vision-Language Recognitioncited by 7
  12. 2025Cluster-Aware Prompt Ensemble Learning for Few-Shot Vision-Language Model Adaptationcited by 7
38 more
  1. 2025Referring Expression Comprehension for Small Objectscited by 6
  2. 2025Medical Knowledge Intervention Prompt Tuning for Medical Image Classificationcited by 5
  3. 2026MMLoP: Multi-Modal Low-Rank Prompting for Efficient Vision-Language Adaptationcited by 5
  4. 2026Reclaiming Lost Text Layers for Source-Free Cross-Domain Few-Shot Learningcited by 5
  5. 2025Prompt-OT: An Optimal Transport Regularization Paradigm for Knowledge Preservation in Vision-Language Model Adaptationcited by 4
  6. 2025FedMVP: Federated Multimodal Visual Prompt Tuning for Vision-Language Modelscited by 4
  7. 2025MMRL++: Parameter-Efficient and Interaction-Aware Representation Learning for Vision-Language Modelscited by 4
  8. 2025Rethinking Few Shot CLIP Benchmarks: A Critical Analysis in the Inductive Settingcited by 4
  9. 2026Mind the Discriminability Trap in Source-Free Cross-domain Few-shot Learningcited by 4
  10. 2024Conditional Prototype Rectification Prompt Learningcited by 3
  11. 2025MGPATH: Vision-Language Model with Multi-Granular Prompt Learning for Few-Shot WSI Classificationcited by 3
  12. 2025Learning Generalizable Prompt for CLIP with Class Similarity Knowledgecited by 3
  13. 2025Generalizable Prompt Learning of CLIP: A Brief Overviewcited by 3
  14. 2025MAO: Efficient Model-Agnostic Optimization of Prompt Tuning for Vision-Language Modelscited by 3
  15. 2025Provably Improving Generalization of Few-Shot Models with Synthetic Datacited by 3
  16. 2025GLAD: Generalizable Tuning for Vision-Language Modelscited by 3
  17. 2025CLIP-SVD: Efficient and Interpretable Vision-Language Adaptation via Singular Valuescited by 3
  18. 2025VaMP: Variational Multi-Modal Prompt Learning for Vision-Language Modelscited by 3
  19. 2024Active Prompt Learning with Vision-Language Model Priorscited by 2
  20. 2025DesCLIP: Robust Continual Learning via General Attribute Descriptions for VLM-Based Visual Recognitioncited by 2
  21. 2025Diversity Covariance-Aware Prompt Learning for Vision-Language Modelscited by 2
  22. 2025Multi-modal Mutual-Guidance Conditional Prompt Learning for Vision-Language Modelscited by 2
  23. 2025Causality-guided Prompt Learning for Vision-language Models via Visual Granulationcited by 2
  24. 2025Global Prompt Refinement with Non-Interfering Attention Masking for One-Shot Federated Learningcited by 2
  25. 2025MoLT: Mixture of Layer-Wise Tokens for Efficient Audio-Visual Learningcited by 2
  26. 2026Guiding Diffusion-based Reconstruction with Contrastive Signals for Balanced Visual Representationcited by 2
  27. 2024Pseudo-Prompt Generating in Pre-trained Vision-Language Models for Multi-Label Medical Image Classificationcited by 1
  28. 2024MePT: Multi-Representation Guided Prompt Tuning for Vision-Language Modelcited by 1
  29. 2025Personalized OVSS: Understanding Personal Concept in Open-Vocabulary Semantic Segmentationcited by 1
  30. 2025Beyond the Seen: Bounded Distribution Estimation for Open-Vocabulary Learningcited by 1
  31. 2025AnchorOPT: Towards Optimizing Dynamic Anchors for Adaptive Prompt Learningcited by 1
  32. 2026bi-modal textual prompt learning for vision-language models in remote sensingcited by 1
  33. 2026DeAR: Fine-Grained VLM Adaptation by Decomposing Attention Head Rolescited by 1
  34. 2026FVG-PT: Adaptive Foreground View-Guided Prompt Tuning for Vision-Language Modelscited by 1
  35. 2024Prompt-based Visual Alignment for Zero-shot Policy Transfercited by 0
  36. 2024LPT: Less-overfitting Prompt Tuning for Vision-Language Modelcited by 0
  37. 2025InPK: Infusing Prior Knowledge into Prompt for Vision-Language Modelscited by 0
  38. 2025ViP2-CLIP: Visual-Perception Prompting with Unified Alignment for Zero-Shot Anomaly Detectioncited by 0

50 most cited of 75 All 75 (JSON)

Similar, not cited

Close in wording, no citation either way

  1. 2024SEP: Self-Enhanced Prompt Tuning for Visual-Language Model
  2. 2023Visual-Language Prompt Tuning with Knowledge-guided Context Optimization
  3. 2024Can Better Text Semantics in Prompt Tuning Improve VLM Generalization?
  4. 2023Task-Oriented Multi-Modal Mutual Leaning for Vision-Language Models
  5. 2023Compound Text-Guided Prompt Tuning via Image-Adaptive Cues
  6. 2023Knowledge-Aware Prompt Tuning for Generalizable Vision-Language Models
  7. 2025A Closer Look at Conditional Prompt Tuning for Vision-Language Models
  8. 2024TextRefiner: Internal Visual Feature as Efficient Refiner for Vision-Language Models Prompt Tuning