mlsanityData to 14 Oct 2026

LoGoPrompt: Synthetic Text Images Can Be Good Visual Prompts for Vision-Language Models

arXiv 2309.01155 ↗First posted 3 Sep 2023cs.CVcited by 29

On the map: Multimodal LLMs / vision-language models → Prompt learning and adaptation of CLIP

Abstract (arXiv)

Prompt engineering is a powerful tool used to enhance the performance of pre-trained models on downstream tasks. For example, providing the prompt "Let's think step by step" improved GPT-3's reasoning accuracy to 63% on MutiArith while prompting "a photo of" filled with a class name enables CLIP to achieve 80% zero-shot accuracy on ImageNet. While previous research has explored prompt learning for the visual modality, analyzing what constitutes a good visual prompt specifically for image recognition is limited. In addition, existing visual prompt tuning methods' generalization ability is worse than text-only prompting tuning. This paper explores our key insight: synthetic text images are good visual prompts for vision-language models! To achieve that, we propose our LoGoPrompt, which reformulates the classification objective to the visual prompt selection and addresses the chicken-and-egg challenge of first adding synthetic text images as class-wise visual prompts or predicting the class first. Without any trainable visual prompt parameters, experimental results on 16 datasets demonstrate that our method consistently outperforms state-of-the-art methods in few-shot learning, base-to-new generalization, and domain generalization.

Key facts. Cited by 29 papers in the map (data through 14 Oct 2026). Placed in Prompt learning and adaptation of CLIP, one of 16 directions in Multimodal LLMs / vision-language models. Builds on 41 of its 55 references that are in the map. First posted on arXiv 3 Sep 2023.

Builds on

41 of 55 references.

Closest ancestors · 7

Same direction, newest first

  1. 2022Prompt-aligned Gradient for Prompt Tuning30 May
  2. 2022Prompt Distribution Learning6 May
  3. 2022Conditional Prompt Learning for Vision-Language Models10 Mar
  4. 2021Tip-Adapter: Training-free CLIP-Adapter for Better Vision-Language Modeling6 Nov
  5. 2021CLIP-Adapter: Better Vision-Language Models with Feature Adapters9 Oct
  6. 2021CPT: Colorful Prompt Tuning for Pre-trained Vision-Language Models24 Sep
  7. 2021Learning to Prompt for Vision-Language Models2 Sep

Other references · 34

  1. 2017Attention Is All You Needcited by 60,361
  2. 2015Deep Residual Learning for Image Recognitioncited by 52,096
  3. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,744
  4. 2020Language Models are Few-Shot Learnerscited by 32,419
  5. 2020An Image is Worth 16x16 Words: Transformers for Image Recognition at Scalecited by 30,186
  6. 2018Representation Learning with Contrastive Predictive Codingcited by 7,777
  7. 2021Zero-Shot Text-to-Image Generationcited by 4,288
  8. 2021Prefix-Tuning: Optimizing Continuous Prompts for Generationcited by 3,915
  9. 2021Scaling Up Visual and Vision-Language Representation Learning With Noisy Text Supervisioncited by 3,864
  10. 2021The Power of Scale for Parameter-Efficient Prompt Tuningcited by 3,795
  11. 2015FaceNet: A Unified Embedding for Face Recognition and Clusteringcited by 2,849
  12. 2021Pre-train, Prompt, and Predict: A Systematic Survey of Prompting Methods in Natural Language Processingcited by 2,234
22 more
  1. 2019ViLBERT: Pretraining Task-Agnostic Visiolinguistic Representations for Vision-and-Language Taskscited by 2,222
  2. 2013Describing Textures in the Wildcited by 2,125
  3. 2020The Many Faces of Robustness: A Critical Analysis of Out-of-Distribution Generalizationcited by 1,850
  4. 2013Fine-Grained Visual Classification of Aircraftcited by 1,726
  5. 2022Visual Prompt Tuningcited by 1,681
  6. 2021Align before Fuse: Vision and Language Representation Learning with Momentum Distillationcited by 1,619
  7. 2019LXMERT: Learning Cross-Modality Encoder Representations from Transformerscited by 1,553
  8. 2019Do ImageNet Classifiers Generalize to ImageNet?cited by 1,515
  9. 2017EuroSAT: A Novel Dataset and Deep Learning Benchmark for Land Use and Land Cover Classificationcited by 1,496
  10. 2019Natural Adversarial Examplescited by 1,408
  11. 2019Learning Robust Global Representations by Penalizing Local Predictive Powercited by 1,013
  12. 2019VL-BERT: Pre-training of Generic Visual-Linguistic Representationscited by 954
  13. 2017In Defense of the Triplet Loss for Person Re-Identificationcited by 786
  14. 2021GPT Understands, Toocited by 716
  15. 2021P-Tuning v2: Prompt Tuning Can Be Comparable to Fine-tuning Universally Across Scales and Taskscited by 689
  16. 2021LiT: Zero-Shot Transfer with Locked-image text Tuningcited by 577
  17. 2021Unifying Vision-and-Language Tasks via Text Generationcited by 412
  18. 2021Supervision Exists Everywhere: A Data Efficient Contrastive Language-Image Pre-training Paradigmcited by 405
  19. 2017Zero-Shot Learning -- The Good, the Bad and the Uglycited by 255
  20. 2021CLOOB: Modern Hopfield Networks with InfoLOOB Outperform CLIPcited by 89
  21. 2022Visual Prompt Tuning for Generative Transfer Learningcited by 75
  22. 2022CLIPPO: Image-and-Language Understanding from Pixels Onlycited by 55

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2023: 5; 2024: 11; 2025: 12; 2026: 1.

29 papers.

Sort:

By area:

  1. 2023DDCoT: Duty-Distinct Chain-of-Thought Prompting for Multimodal Reasoning in Language Modelscited by 128
  2. 2023Free-Bloom: Zero-Shot Text-to-Video Generator with LLM Director and LDM Animatorcited by 74
  3. 2024Dual Memory Networks: A Versatile Adaptation Approach for Vision-Language Modelscited by 52
  4. 2024AWT: Transferring Vision-Language Models via Augmentation, Weighting, and Transportationcited by 28
  5. 2024Improving Zero-shot Generalization of Learned Prompts via Unsupervised Knowledge Distillationcited by 18
  6. 2023Parrot Captions Teach CLIP to Spot Textcited by 17
  7. 2025Vision Function Layer in Multimodal LLMscited by 17
  8. 2025Rethinking Query-based Transformer for Continual Image Segmentationcited by 14
  9. 2024Part2Object: Hierarchical Unsupervised 3D Instance Segmentationcited by 12
  10. 2023Diversified in-domain synthesis with efficient fine-tuning for few-shot classificationcited by 11
  11. 2024The devil is in the object boundary: towards annotation-free instance segmentation using Foundation Modelscited by 10
  12. 2024Efficient Test-Time Prompt Tuning for Vision-Language Modelscited by 10
17 more
  1. 2025Why LVLMs Are More Prone to Hallucinations in Longer Responses: The Role of Contextcited by 10
  2. 2024Tree of Attributes Prompt Learning for Vision-Language Modelscited by 7
  3. 2023Descriptor and Word Soups: Overcoming the Parameter Efficiency Accuracy Tradeoff for Out-of-Distribution Few-shot Learningcited by 6
  4. 2025What Changed and What Could Have Changed? State-Change Counterfactuals for Procedure-Aware Video Representation Learningcited by 6
  5. 2025Intervene-All-Paths: Unified Mitigation of LVLM Hallucinations across Alignment Formatscited by 6
  6. 2026MMLoP: Multi-Modal Low-Rank Prompting for Efficient Vision-Language Adaptationcited by 5
  7. 2025Sim-DETR: Unlock DETR for Temporal Sentence Groundingcited by 4
  8. 2024Exploring the Potential of Synthetic Data to Replace Real Datacited by 3
  9. 2025No More Sibling Rivalry: Debiasing Human-Object Interaction Detectioncited by 3
  10. 2025Visual Adaptive Prompting for Compositional Zero-Shot Learningcited by 2
  11. 2025Causality-guided Prompt Learning for Vision-language Models via Visual Granulationcited by 2
  12. 2024Craft: Cross-modal Aligned Features Improve Robustness of Prompt Tuningcited by 1
  13. 2025Augmenting Moment Retrieval: Zero-Dependency Two-Stage Learningcited by 1
  14. 2024RESTORE: Towards Feature Shift for Vision-Language Prompt Learningcited by 0
  15. 2024Can Better Text Semantics in Prompt Tuning Improve VLM Generalization?cited by 0
  16. 2025FedMGP: Personalized Federated Learning with Multi-Group Text-Visual Promptscited by 0
  17. 2025Decoupling Augmentation Bias in Prompt Learning for Vision-Language Modelscited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2023A Systematic Survey of Prompt Engineering on Vision-Language Foundation Models
  2. 2022Prompting through Prototype: A Prototype-based Prompt Learning on Pretrained Vision-Language Models
  3. 2024Generalizable Prompt Tuning for Vision-Language Models
  4. 2023PRE: Vision-Language Prompt Learning with Reparameterization Encoder
  5. 2022Learning Domain Invariant Prompt for Vision-Language Models
  6. 2022Test-Time Prompt Tuning for Zero-Shot Generalization in Vision-Language Models
  7. 2023Towards Robust Prompts on Vision-Language Models
  8. 2023LaViP:Language-Grounded Visual Prompts