# Texts as Images in Prompt Tuning for Multi-Label Image Recognition

arXiv: https://arxiv.org/abs/2211.12739 · HTML: https://mlsanity.com/paper/2211.12739/

## Key facts

- Cited by 51 papers in the map (data through 14 Oct 2026).
- Placed in Prompt learning and adaptation of CLIP, one of 16 directions in Multimodal LLMs / vision-language models.
- Builds on 43 of its 61 references that are in the map.
- First posted on arXiv 23 Nov 2022.

## Placed in

- Multimodal LLMs / vision-language models / [Prompt learning and adaptation of CLIP](https://mlsanity.com/mllm/prompt-learning-and-adaptation-of-clip/)

## Builds on (most cited first, 30 of 43)

- [Learning Transferable Visual Models From Natural Language Supervision](https://mlsanity.com/paper/2103.00020/) (2021)
- [Microsoft COCO: Common Objects in Context](https://mlsanity.com/paper/1405.0312/) (2014)
- [Flamingo: a Visual Language Model for Few-Shot Learning](https://mlsanity.com/paper/2204.14198/) (2022)
- [Scaling Up Visual and Vision-Language Representation Learning With Noisy Text Supervision](https://mlsanity.com/paper/2102.05918/) (2021)
- [Learning to Prompt for Vision-Language Models](https://mlsanity.com/paper/2109.01134/) (2021)
- [Visual Prompt Tuning](https://mlsanity.com/paper/2203.12119/) (2022)
- [Conditional Prompt Learning for Vision-Language Models](https://mlsanity.com/paper/2203.05557/) (2022)
- [CLIP-Adapter: Better Vision-Language Models with Feature Adapters](https://mlsanity.com/paper/2110.04544/) (2021)
- [Florence: A New Foundation Model for Computer Vision](https://mlsanity.com/paper/2111.11432/) (2021)
- [FILIP: Fine-grained Interactive Language-Image Pre-Training](https://mlsanity.com/paper/2111.07783/) (2021)
- [GroupViT: Semantic Segmentation Emerges from Text Supervision](https://mlsanity.com/paper/2202.11094/) (2022)
- [Supervision Exists Everywhere: A Data Efficient Contrastive Language-Image Pre-training Paradigm](https://mlsanity.com/paper/2110.05208/) (2021)
- [Test-Time Prompt Tuning for Zero-Shot Generalization in Vision-Language Models](https://mlsanity.com/paper/2209.07511/) (2022)
- [Asymmetric Loss For Multi-Label Classification](https://mlsanity.com/paper/2009.14119/) (2020)
- [Tip-Adapter: Training-free Adaption of CLIP for Few-shot Classification](https://mlsanity.com/paper/2207.09519/) (2022)
- [Prompt-aligned Gradient for Prompt Tuning](https://mlsanity.com/paper/2205.14865/) (2022)
- [Multi-Label Image Recognition with Graph Convolutional Networks](https://mlsanity.com/paper/1904.03582/) (2019)
- [Prompt Distribution Learning](https://mlsanity.com/paper/2205.03340/) (2022)
- [CNN-RNN: A Unified Framework for Multi-label Image Classification](https://mlsanity.com/paper/1604.04573/) (2016)
- [CPT: Colorful Prompt Tuning for Pre-trained Vision-Language Models](https://mlsanity.com/paper/2109.11797/) (2021)
- [Unified Vision and Language Prompt Learning](https://mlsanity.com/paper/2210.07225/) (2022)
- [Neural Prompt Search](https://mlsanity.com/paper/2206.04673/) (2022)
- [Domain Adaptation via Prompt Learning](https://mlsanity.com/paper/2202.06687/) (2022)
- [Unsupervised Prompt Learning for Vision-Language Models](https://mlsanity.com/paper/2204.03649/) (2022)
- [Learning a Deep ConvNet for Multi-label Classification with Partial Labels](https://mlsanity.com/paper/1902.09720/) (2019)
- [Learning Semantic-Specific Graph Representation for Multi-Label Image Recognition](https://mlsanity.com/paper/1908.07325/) (2019)
- [DualCoOp: Fast Adaptation to Multi-Label Recognition with Limited Annotations](https://mlsanity.com/paper/2206.09541/) (2022)
- [Attention-Driven Dynamic Graph Convolutional Network for Multi-Label Image Recognition](https://mlsanity.com/paper/2012.02994/) (2020)
- [Cross-Modality Attention with Semantic Graph Embedding for Multi-Label Classification](https://mlsanity.com/paper/1912.07872/) (2019)
- [Knowledge-Guided Multi-Label Few-Shot Learning for General Image Recognition](https://mlsanity.com/paper/2009.09450/) (2020)

## Built on by (most cited first, 30 of 51)

- [Vision-Language Models for Vision Tasks: A Survey](https://mlsanity.com/paper/2304.00685/) (2023)
- [What does CLIP know about a red circle? Visual prompt engineering for VLMs](https://mlsanity.com/paper/2304.06712/) (2023)
- [A Closer Look at the Explainability of Contrastive Language-Image Pre-training](https://mlsanity.com/paper/2304.05653/) (2023)
- [Learning Mask-aware CLIP Representations for Zero-Shot Segmentation](https://mlsanity.com/paper/2310.00240/) (2023)
- [Bayesian Prompt Flow Learning for Zero-Shot Anomaly Detection](https://mlsanity.com/paper/2503.10080/) (2025)
- [AWT: Transferring Vision-Language Models via Augmentation, Weighting, and Transportation](https://mlsanity.com/paper/2407.04603/) (2024)
- [ViLa-MIL: Dual-scale Vision-Language Multiple Instance Learning for Whole Slide Image Classification](https://mlsanity.com/paper/2502.08391/) (2025)
- [Attention Prompting on Image for Large Vision-Language Models](https://mlsanity.com/paper/2409.17143/) (2024)
- [Advances in Multimodal Adaptation and Generalization: From Traditional Approaches to Foundation Models](https://mlsanity.com/paper/2501.18592/) (2025)
- [DualCoOp++: Fast and Effective Adaptation to Multi-Label Recognition with Limited Annotations](https://mlsanity.com/paper/2308.01890/) (2023)
- [Can Multimodal Large Language Models Understand Spatial Relations?](https://mlsanity.com/paper/2505.19015/) (2025)
- [TagCLIP: A Local-to-Global Framework to Enhance Open-Vocabulary Multi-Label Classification of CLIP Without Training](https://mlsanity.com/paper/2312.12828/) (2023)
- [LMPT: Prompt Tuning with Class-Specific Embedding Loss for Long-tailed Multi-Label Visual Recognition](https://mlsanity.com/paper/2305.04536/) (2023)
- [TAI++: Text as Image for Multi-Label Image Classification by Co-Learning Transferable Prompt](https://mlsanity.com/paper/2405.06926/) (2024)
- [Learning to Prompt Segment Anything Models](https://mlsanity.com/paper/2401.04651/) (2024)
- [Text-Region Matching for Multi-Label Image Recognition with Missing Labels](https://mlsanity.com/paper/2407.18520/) (2024)
- [Data-free Multi-label Image Recognition via LLM-powered Prompt Tuning](https://mlsanity.com/paper/2403.01209/) (2024)
- [FALIP: Visual Prompt as Foveal Attention Boosts CLIP Zero-Shot Performance](https://mlsanity.com/paper/2407.05578/) (2024)
- [Compound Text-Guided Prompt Tuning via Image-Adaptive Cues](https://mlsanity.com/paper/2312.06401/) (2023)
- [Seeing Beyond Classes: Zero-Shot Grounded Situation Recognition via Language Explainer](https://mlsanity.com/paper/2404.15785/) (2024)
- [Decouple before Align: Visual Disentanglement Enhances Prompt Tuning](https://mlsanity.com/paper/2508.00395/) (2025)
- [MRAD: Zero-Shot Anomaly Detection with Memory-Driven Retrieval](https://mlsanity.com/paper/2602.00522/) (2026)
- [TTD: Text-Tag Self-Distillation Enhancing Image-Text Alignment in CLIP to Alleviate Single Tag Bias](https://mlsanity.com/paper/2404.00384/) (2024)
- [Zero-shot Compositional Action Recognition with Neural Logic Constraints](https://mlsanity.com/paper/2508.02320/) (2025)
- [Text as Image: Learning Transferable Adapter for Multi-Label Classification](https://mlsanity.com/paper/2312.04160/) (2023)
- [Text as Any-Modality for Zero-Shot Classification by Consistent Prompt Tuning](https://mlsanity.com/paper/2508.06382/) (2025)
- [Query-Based Knowledge Sharing for Open-Vocabulary Multi-Label Classification](https://mlsanity.com/paper/2401.01181/) (2024)
- [PVLR: Prompt-driven Visual-Linguistic Representation Learning for Multi-Label Image Recognition](https://mlsanity.com/paper/2401.17881/) (2024)
- [Analyzing the Roles of Language and Vision in Learning from Limited Data](https://mlsanity.com/paper/2403.19669/) (2024)
- [SSPA: Split-and-Synthesize Prompting with Gated Alignments for Multi-Label Image Recognition](https://mlsanity.com/paper/2407.20920/) (2024)

## Abstract (arXiv)

Prompt tuning has been employed as an efficient way to adapt large vision-language pre-trained models (e.g. CLIP) to various downstream tasks in data-limited or label-limited settings. Nonetheless, visual data (e.g., images) is by default prerequisite for learning prompts in existing methods. In this work, we advocate that the effectiveness of image-text contrastive learning in aligning the two modalities (for training CLIP) further makes it feasible to treat texts as images for prompt tuning and introduce TaI prompting. In contrast to the visual data, text descriptions are easy to collect, and their class labels can be directly derived. Particularly, we apply TaI prompting to multi-label image recognition, where sentences in the wild serve as alternatives to images for prompt tuning. Moreover, with TaI, double-grained prompt tuning (TaI-DPT) is further presented to extract both coarse-grained and fine-grained embeddings for enhancing the multi-label recognition performance. Experimental results show that our proposed TaI-DPT outperforms zero-shot CLIP by a large margin on multiple benchmarks, e.g., MS-COCO, VOC2007, and NUS-WIDE, while it can be combined with existing methods of prompting from images to improve recognition performance further. Code is released at https://github.com/guozix/TaI-DPT.