# TCP:Textual-based Class-aware Prompt tuning for Visual-Language Model

arXiv: https://arxiv.org/abs/2311.18231 · HTML: https://mlsanity.com/paper/2311.18231/

## Key facts

- Cited by 76 papers in the map (data through 14 Oct 2026).
- Placed in Prompt learning and adaptation of CLIP, one of 16 directions in Multimodal LLMs / vision-language models.
- Builds on 20 of its 52 references that are in the map.
- First posted on arXiv 30 Nov 2023.

## Placed in

- Multimodal LLMs / vision-language models / [Prompt learning and adaptation of CLIP](https://mlsanity.com/mllm/prompt-learning-and-adaptation-of-clip/)

## Builds on (most cited first, 20 of 20)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [Learning Transferable Visual Models From Natural Language Supervision](https://mlsanity.com/paper/2103.00020/) (2021)
- [An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale](https://mlsanity.com/paper/2010.11929/) (2020)
- [LoRA: Low-Rank Adaptation of Large Language Models](https://mlsanity.com/paper/2106.09685/) (2021)
- [Flamingo: a Visual Language Model for Few-Shot Learning](https://mlsanity.com/paper/2204.14198/) (2022)
- [BLIP: Bootstrapping Language-Image Pre-training for Unified Vision-Language Understanding and Generation](https://mlsanity.com/paper/2201.12086/) (2022)
- [LAION-5B: An open large-scale dataset for training next generation image-text models](https://mlsanity.com/paper/2210.08402/) (2022)
- [Scaling Up Visual and Vision-Language Representation Learning With Noisy Text Supervision](https://mlsanity.com/paper/2102.05918/) (2021)
- [ViLBERT: Pretraining Task-Agnostic Visiolinguistic Representations for Vision-and-Language Tasks](https://mlsanity.com/paper/1908.02265/) (2019)
- [Fine-Grained Visual Classification of Aircraft](https://mlsanity.com/paper/1306.5151/) (2013)
- [LAION-400M: Open Dataset of CLIP-Filtered 400 Million Image-Text Pairs](https://mlsanity.com/paper/2111.02114/) (2021)
- [ViLT: Vision-and-Language Transformer Without Convolution or Region Supervision](https://mlsanity.com/paper/2102.03334/) (2021)
- [CLIP-Adapter: Better Vision-Language Models with Feature Adapters](https://mlsanity.com/paper/2110.04544/) (2021)
- [Towards a Unified View of Parameter-Efficient Transfer Learning](https://mlsanity.com/paper/2110.04366/) (2021)
- [Unified Vision and Language Prompt Learning](https://mlsanity.com/paper/2210.07225/) (2022)
- [PLOT: Prompt Learning with Optimal Transport for Vision-Language Models](https://mlsanity.com/paper/2210.01253/) (2022)
- [DePT: Decoupled Prompt Tuning](https://mlsanity.com/paper/2309.07439/) (2023)
- [Multitask Vision-Language Prompt Tuning](https://mlsanity.com/paper/2211.11720/) (2022)
- [Gradient-Regulated Meta-Prompt Learning for Generalizable Vision-Language Models](https://mlsanity.com/paper/2303.06571/) (2023)
- [Prompt Tuning based Adapter for Vision-Language Model Adaption](https://mlsanity.com/paper/2303.15234/) (2023)

## Built on by (most cited first, 30 of 76)

- [Parameter-Efficient Fine-Tuning for Foundation Models](https://mlsanity.com/paper/2501.13787/) (2025)
- [AWT: Transferring Vision-Language Models via Augmentation, Weighting, and Transportation](https://mlsanity.com/paper/2407.04603/) (2024)
- [MAP-VLA: Memory-Augmented Prompting for Vision-Language-Action Model in Robotic Manipulation](https://mlsanity.com/paper/2511.09516/) (2025)
- [DPC: Dual-Prompt Collaboration for Tuning Vision-Language Models](https://mlsanity.com/paper/2503.13443/) (2025)
- [OpenworldAUC: Towards Unified Evaluation and Optimization for Open-world Prompt Tuning](https://mlsanity.com/paper/2505.05180/) (2025)
- [Hierarchical Cross-modal Prompt Learning for Vision-Language Models](https://mlsanity.com/paper/2507.14976/) (2025)
- [Skip Tuning: Pre-trained Vision-Language Models are Effective and Efficient Adapters Themselves](https://mlsanity.com/paper/2412.11509/) (2024)
- [Mettle: Meta-Token Learning for Memory-Efficient Audio-Visual Adaptation](https://mlsanity.com/paper/2506.23271/) (2025)
- [Understanding and Mitigating Miscalibration in Prompt Tuning for Vision-Language Models](https://mlsanity.com/paper/2410.02681/) (2024)
- [DA-Ada: Learning Domain-Aware Adapter for Domain Adaptive Object Detection](https://mlsanity.com/paper/2410.09004/) (2024)
- [Black Sheep in the Herd: Playing with Spuriously Correlated Attributes for Vision-Language Recognition](https://mlsanity.com/paper/2502.15809/) (2025)
- [Cluster-Aware Prompt Ensemble Learning for Few-Shot Vision-Language Model Adaptation](https://mlsanity.com/paper/2510.09867/) (2025)
- [Referring Expression Comprehension for Small Objects](https://mlsanity.com/paper/2510.03701/) (2025)
- [Medical Knowledge Intervention Prompt Tuning for Medical Image Classification](https://mlsanity.com/paper/2511.12639/) (2025)
- [MMLoP: Multi-Modal Low-Rank Prompting for Efficient Vision-Language Adaptation](https://mlsanity.com/paper/2602.21397/) (2026)
- [Reclaiming Lost Text Layers for Source-Free Cross-Domain Few-Shot Learning](https://mlsanity.com/paper/2603.05235/) (2026)
- [Prompt-OT: An Optimal Transport Regularization Paradigm for Knowledge Preservation in Vision-Language Model Adaptation](https://mlsanity.com/paper/2503.08906/) (2025)
- [FedMVP: Federated Multimodal Visual Prompt Tuning for Vision-Language Models](https://mlsanity.com/paper/2504.20860/) (2025)
- [MMRL++: Parameter-Efficient and Interaction-Aware Representation Learning for Vision-Language Models](https://mlsanity.com/paper/2505.10088/) (2025)
- [Rethinking Few Shot CLIP Benchmarks: A Critical Analysis in the Inductive Setting](https://mlsanity.com/paper/2507.20834/) (2025)
- [Mind the Discriminability Trap in Source-Free Cross-domain Few-shot Learning](https://mlsanity.com/paper/2603.13341/) (2026)
- [Conditional Prototype Rectification Prompt Learning](https://mlsanity.com/paper/2404.09872/) (2024)
- [MGPATH: Vision-Language Model with Multi-Granular Prompt Learning for Few-Shot WSI Classification](https://mlsanity.com/paper/2502.07409/) (2025)
- [Learning Generalizable Prompt for CLIP with Class Similarity Knowledge](https://mlsanity.com/paper/2502.11969/) (2025)
- [Generalizable Prompt Learning of CLIP: A Brief Overview](https://mlsanity.com/paper/2503.01263/) (2025)
- [MAO: Efficient Model-Agnostic Optimization of Prompt Tuning for Vision-Language Models](https://mlsanity.com/paper/2503.18160/) (2025)
- [Provably Improving Generalization of Few-Shot Models with Synthetic Data](https://mlsanity.com/paper/2505.24190/) (2025)
- [GLAD: Generalizable Tuning for Vision-Language Models](https://mlsanity.com/paper/2507.13089/) (2025)
- [CLIP-SVD: Efficient and Interpretable Vision-Language Adaptation via Singular Values](https://mlsanity.com/paper/2509.03740/) (2025)
- [VaMP: Variational Multi-Modal Prompt Learning for Vision-Language Models](https://mlsanity.com/paper/2511.22664/) (2025)

## Abstract (arXiv)

Prompt tuning represents a valuable technique for adapting pre-trained visual-language models (VLM) to various downstream tasks. Recent advancements in CoOp-based methods propose a set of learnable domain-shared or image-conditional textual tokens to facilitate the generation of task-specific textual classifiers. However, those textual tokens have a limited generalization ability regarding unseen domains, as they cannot dynamically adjust to the distribution of testing classes. To tackle this issue, we present a novel Textual-based Class-aware Prompt tuning(TCP) that explicitly incorporates prior knowledge about classes to enhance their discriminability. The critical concept of TCP involves leveraging Textual Knowledge Embedding (TKE) to map the high generalizability of class-level textual knowledge into class-aware textual tokens. By seamlessly integrating these class-aware prompts into the Text Encoder, a dynamic class-aware classifier is generated to enhance discriminability for unseen domains. During inference, TKE dynamically generates class-aware prompts related to the unseen classes. Comprehensive evaluations demonstrate that TKE serves as a plug-and-play module effortlessly combinable with existing methods. Furthermore, TCP consistently achieves superior performance while demanding less training time. Code:https://github.com/htyao89/Textual-based_Class-aware_prompt_tuning/