mlsanityData to 7 Oct 2026

Texts as Images in Prompt Tuning for Multi-Label Image Recognition

arXiv 2211.12739 ↗First posted 23 Nov 2022cs.CVcited by 51

On the map: Multimodal LLMs / vision-language models → Prompt learning and adaptation of CLIP

Abstract (arXiv)

Prompt tuning has been employed as an efficient way to adapt large vision-language pre-trained models (e.g. CLIP) to various downstream tasks in data-limited or label-limited settings. Nonetheless, visual data (e.g., images) is by default prerequisite for learning prompts in existing methods. In this work, we advocate that the effectiveness of image-text contrastive learning in aligning the two modalities (for training CLIP) further makes it feasible to treat texts as images for prompt tuning and introduce TaI prompting. In contrast to the visual data, text descriptions are easy to collect, and their class labels can be directly derived. Particularly, we apply TaI prompting to multi-label image recognition, where sentences in the wild serve as alternatives to images for prompt tuning. Moreover, with TaI, double-grained prompt tuning (TaI-DPT) is further presented to extract both coarse-grained and fine-grained embeddings for enhancing the multi-label recognition performance. Experimental results show that our proposed TaI-DPT outperforms zero-shot CLIP by a large margin on multiple benchmarks, e.g., MS-COCO, VOC2007, and NUS-WIDE, while it can be combined with existing methods of prompting from images to improve recognition performance further. Code is released at https://github.com/guozix/TaI-DPT.

Key facts. Cited by 51 papers in the map (data through 7 Oct 2026). Placed in Prompt learning and adaptation of CLIP, one of 16 directions in Multimodal LLMs / vision-language models. Builds on 43 of its 61 references that are in the map. First posted on arXiv 23 Nov 2022.

Builds on

43 of 61 references.

Closest ancestors · 11

Same direction, newest first

  1. 2022Prompting through Prototype: A Prototype-based Prompt Learning on Pretrained Vision-Language Models19 Oct
  2. 2022Unified Vision and Language Prompt Learning13 Oct
  3. 2022Test-Time Prompt Tuning for Zero-Shot Generalization in Vision-Language Models15 Sep
  4. 2022Tip-Adapter: Training-free Adaption of CLIP for Few-shot Classification19 Jul
  5. 2022Prompt-aligned Gradient for Prompt Tuning30 May
  6. 2022Prompt Distribution Learning6 May
  7. 2022Unsupervised Prompt Learning for Vision-Language Models7 Apr
  8. 2022Conditional Prompt Learning for Vision-Language Models10 Mar
  9. 2021CLIP-Adapter: Better Vision-Language Models with Feature Adapters9 Oct
  10. 2021CPT: Colorful Prompt Tuning for Pre-trained Vision-Language Models24 Sep
  11. 2021Learning to Prompt for Vision-Language Models2 Sep

Other references · 32

  1. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,456
  2. 2014Microsoft COCO: Common Objects in Contextcited by 17,498
  3. 2022Flamingo: a Visual Language Model for Few-Shot Learningcited by 4,337
  4. 2021Scaling Up Visual and Vision-Language Representation Learning With Noisy Text Supervisioncited by 3,848
  5. 2022Visual Prompt Tuningcited by 1,674
  6. 2021Florence: A New Foundation Model for Computer Visioncited by 802
  7. 2021FILIP: Fine-grained Interactive Language-Image Pre-Trainingcited by 626
  8. 2022GroupViT: Semantic Segmentation Emerges from Text Supervisioncited by 529
  9. 2021Supervision Exists Everywhere: A Data Efficient Contrastive Language-Image Pre-training Paradigmcited by 405
  10. 2020Asymmetric Loss For Multi-Label Classificationcited by 379
  11. 2019Multi-Label Image Recognition with Graph Convolutional Networkscited by 320
  12. 2016CNN-RNN: A Unified Framework for Multi-label Image Classificationcited by 189
20 more
  1. 2022Neural Prompt Searchcited by 133
  2. 2022Domain Adaptation via Prompt Learningcited by 128
  3. 2019Learning a Deep ConvNet for Multi-label Classification with Partial Labelscited by 124
  4. 2019Learning Semantic-Specific Graph Representation for Multi-Label Image Recognitioncited by 116
  5. 2022DualCoOp: Fast Adaptation to Multi-Label Recognition with Limited Annotationscited by 102
  6. 2020Attention-Driven Dynamic Graph Convolutional Network for Multi-Label Image Recognitioncited by 80
  7. 2019Cross-Modality Attention with Semantic Graph Embedding for Multi-Label Classificationcited by 78
  8. 2020Knowledge-Guided Multi-Label Few-Shot Learning for General Image Recognitioncited by 69
  9. 2014CNN: Single-label to Multi-labelcited by 62
  10. 2013Deep Convolutional Ranking for Multilabel Image Annotationcited by 47
  11. 2021Structured Semantic Transfer for Multi-Label Recognition with Partial Labelscited by 41
  12. 2015Exploit Bounding Box Annotations for Multi-label Object Recognitioncited by 39
  13. 2019LaSO: Label-Set Operations networks for multi-label few-shot learningcited by 38
  14. 2018Multi-Label Image Classification via Knowledge Distillation from Weakly-Supervised Detectioncited by 37
  15. 2018Multi-Evidence Filtering and Fusion for Multi-Label Classification, Object Detection and Semantic Segmentation Based on Weakly Supervised Learningcited by 33
  16. 2022Open-Vocabulary Multi-Label Classification via Multi-Modal Knowledge Transfercited by 33
  17. 2022Semantic-Aware Representation Blending for Multi-Label Image Recognition with Partial Labelscited by 29
  18. 2016Multi-Label Image Classification with Regional Latent Semantic Dependenciescited by 23
  19. 2021Meta-Learning for Multi-Label Few-Shot Classificationcited by 21
  20. 2022All Mistakes Are Not Equal: Comprehensive Hierarchy Aware Multi-label Predictions (CHAMP)cited by 3

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2023: 10; 2024: 21; 2025: 14; 2026: 6.

51 papers.

Sort:

By area:

  1. 2023Vision-Language Models for Vision Tasks: A Surveycited by 834
  2. 2023What does CLIP know about a red circle? Visual prompt engineering for VLMscited by 215
  3. 2023A Closer Look at the Explainability of Contrastive Language-Image Pre-trainingcited by 95
  4. 2023Learning Mask-aware CLIP Representations for Zero-Shot Segmentationcited by 57
  5. 2025Bayesian Prompt Flow Learning for Zero-Shot Anomaly Detectioncited by 31
  6. 2025ViLa-MIL: Dual-scale Vision-Language Multiple Instance Learning for Whole Slide Image Classificationcited by 28
  7. 2024AWT: Transferring Vision-Language Models via Augmentation, Weighting, and Transportationcited by 27
  8. 2024Attention Prompting on Image for Large Vision-Language Modelscited by 23
  9. 2025Advances in Multimodal Adaptation and Generalization: From Traditional Approaches to Foundation Modelscited by 23
  10. 2023DualCoOp++: Fast and Effective Adaptation to Multi-Label Recognition with Limited Annotationscited by 22
  11. 2025Can Multimodal Large Language Models Understand Spatial Relations?cited by 20
  12. 2023TagCLIP: A Local-to-Global Framework to Enhance Open-Vocabulary Multi-Label Classification of CLIP Without Trainingcited by 19
38 more
  1. 2023LMPT: Prompt Tuning with Class-Specific Embedding Loss for Long-tailed Multi-Label Visual Recognitioncited by 15
  2. 2024TAI++: Text as Image for Multi-Label Image Classification by Co-Learning Transferable Promptcited by 14
  3. 2024Learning to Prompt Segment Anything Modelscited by 13
  4. 2024Text-Region Matching for Multi-Label Image Recognition with Missing Labelscited by 9
  5. 2024Data-free Multi-label Image Recognition via LLM-powered Prompt Tuningcited by 8
  6. 2023Compound Text-Guided Prompt Tuning via Image-Adaptive Cuescited by 7
  7. 2024Seeing Beyond Classes: Zero-Shot Grounded Situation Recognition via Language Explainercited by 7
  8. 2024FALIP: Visual Prompt as Foveal Attention Boosts CLIP Zero-Shot Performancecited by 7
  9. 2025Decouple before Align: Visual Disentanglement Enhances Prompt Tuningcited by 7
  10. 2026MRAD: Zero-Shot Anomaly Detection with Memory-Driven Retrievalcited by 5
  11. 2024TTD: Text-Tag Self-Distillation Enhancing Image-Text Alignment in CLIP to Alleviate Single Tag Biascited by 4
  12. 2025Zero-shot Compositional Action Recognition with Neural Logic Constraintscited by 4
  13. 2023Text as Image: Learning Transferable Adapter for Multi-Label Classificationcited by 3
  14. 2025Text as Any-Modality for Zero-Shot Classification by Consistent Prompt Tuningcited by 3
  15. 2024Query-Based Knowledge Sharing for Open-Vocabulary Multi-Label Classificationcited by 2
  16. 2024PVLR: Prompt-driven Visual-Linguistic Representation Learning for Multi-Label Image Recognitioncited by 2
  17. 2024Analyzing the Roles of Language and Vision in Learning from Limited Datacited by 2
  18. 2024SSPA: Split-and-Synthesize Prompting with Gated Alignments for Multi-Label Image Recognitioncited by 2
  19. 2025SPARC: Score Prompting and Adaptive Fusion for Zero-Shot Multi-Label Recognition in Vision-Language Modelscited by 2
  20. 2025Dual-View Alignment Learning with Hierarchical-Prompt for Class-Imbalance Multi-Label Classificationcited by 2
  21. 2023Language-assisted Vision Model Debugger: A Sample-Free Approach to Finding and Fixing Bugscited by 1
  22. 2024Rethinking Prompting Strategies for Multi-Label Recognition with Partial Annotationscited by 1
  23. 2024Debiasing Vison-Language Models with Text-Only Trainingcited by 1
  24. 2025Efficiently Disentangling CLIP for Multi-Object Perceptioncited by 1
  25. 2025Text to Image for Multi-Label Image Recognition with Joint Prompt-Adapter Learningcited by 1
  26. 2025Unleashing the Power of Vision-Language Models for Long-Tailed Multi-Label Visual Recognitioncited by 1
  27. 2026MUSE: Harnessing Precise and Diverse Semantics for Few-Shot Whole Slide Image Classificationcited by 1
  28. 2024RESTORE: Towards Feature Shift for Vision-Language Prompt Learningcited by 0
  29. 2024Optimization of Prompt Learning via Multi-Knowledge Representation for Vision-Language Modelscited by 0
  30. 2024A Survey on Incomplete Multi-label Learning: Recent Advances and Future Trendscited by 0
  31. 2024The Solution for Language-Enhanced Image New Category Discoverycited by 0
  32. 2024Textual Training for the Hassle-Free Removal of Unwanted Visual Data: Case Studies on OOD and Hateful Image Detectioncited by 0
  33. 2024Modeling Multi-modal Cross-interaction for Multi-label Few-shot Image Classification Based on Local Feature Selectioncited by 0
  34. 2025RWKV-UI: UI Understanding with Enhanced Perception and Reasoningcited by 0
  35. 2025Incomplete Multi-Label Image Recognition by Co-learning Semantic-Aware Features and Label Recoverycited by 0
  36. 2026D2ACE: Multi-Label Batch Selection Guided by Dual Dynamics and Adaptive Correlation Enhancementcited by 0
  37. 2026Rethinking Multi-Label Image Classification With Deep Learning: Taxonomy, Challenge, and Outlookcited by 0
  38. 2026Adapting Dense Vision-Language Relationships for Multi-label Classification with Partial Labelcited by 0

50 most cited of 51 All 51 (JSON)

Similar, not cited

Close in wording, no citation either way

  1. 2023Task-Oriented Multi-Modal Mutual Leaning for Vision-Language Models
  2. 2023Why Is Prompt Tuning for Vision-Language Models Robust to Noisy Labels?
  3. 2024Tuning Vision-Language Models with Candidate Labels by Prompt Alignment
  4. 2024Can Better Text Semantics in Prompt Tuning Improve VLM Generalization?
  5. 2025Correlative and Discriminative Label Grouping for Multi-Label Visual Prompt Tuning
  6. 2023Tuning Multi-mode Token-level Prompt Alignment across Modalities
  7. 2026Structured-Condensed Prompt Tuning in Vision-Language Models for Fine-grained Image Recognition
  8. 2025Dual Prompt Learning for Adapting Vision-Language Models to Downstream Image-Text Retrieval