mlsanityData to 7 Oct 2026

Test-Time Prompt Tuning for Zero-Shot Generalization in Vision-Language Models

arXiv 2209.07511 ↗First posted 15 Sep 2022cs.CVcited by 379

On the map: Multimodal LLMs / vision-language models → Prompt learning and adaptation of CLIP

Abstract (arXiv)

Pre-trained vision-language models (e.g., CLIP) have shown promising zero-shot generalization in many downstream tasks with properly designed text prompts. Instead of relying on hand-engineered prompts, recent works learn prompts using the training data from downstream tasks. While effective, training on domain-specific data reduces a model's generalization capability to unseen new domains. In this work, we propose test-time prompt tuning (TPT), a method that can learn adaptive prompts on the fly with a single test sample. For image classification, TPT optimizes the prompt by minimizing the entropy with confidence selection so that the model has consistent predictions across different augmented views of each test sample. In evaluating generalization to natural distribution shifts, TPT improves the zero-shot top-1 accuracy of CLIP by 3.6% on average, surpassing previous prompt tuning approaches that require additional task-specific training data. In evaluating cross-dataset generalization with unseen categories, TPT performs on par with the state-of-the-art approaches that use additional training data. Project page: https://azshue.github.io/TPT.

Key facts. Cited by 379 papers in the map (data through 7 Oct 2026). Placed in Prompt learning and adaptation of CLIP, one of 16 directions in Multimodal LLMs / vision-language models. Builds on 58 of its 79 references that are in the map. First posted on arXiv 15 Sep 2022.

Builds on

58 of 79 references.

Closest ancestors · 5

Same direction, newest first

  1. 2022Unsupervised Prompt Learning for Vision-Language Models7 Apr
  2. 2022Conditional Prompt Learning for Vision-Language Models10 Mar
  3. 2021VT-CLIP: Enhancing Vision-Language Models with Visual-guided Texts4 Dec
  4. 2021Tip-Adapter: Training-free CLIP-Adapter for Better Vision-Language Modeling6 Nov
  5. 2021Learning to Prompt for Vision-Language Models2 Sep

Other references · 53

  1. 2015Deep Residual Learning for Image Recognitioncited by 51,809
  2. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,204
  3. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,456
  4. 2020Language Models are Few-Shot Learnerscited by 32,199
  5. 2020A Simple Framework for Contrastive Learning of Visual Representationscited by 11,979
  6. 2022Hierarchical Text-Conditional Image Generation with CLIP Latentscited by 6,539
  7. 2017Model-Agnostic Meta-Learning for Fast Adaptation of Deep Networkscited by 4,861
  8. 2022BLIP: Bootstrapping Language-Image Pre-training for Unified Vision-Language Understanding and Generationcited by 4,310
  9. 2021On the Opportunities and Risks of Foundation Modelscited by 3,961
  10. 2021Prefix-Tuning: Optimizing Continuous Prompts for Generationcited by 3,878
  11. 2021Scaling Up Visual and Vision-Language Representation Learning With Noisy Text Supervisioncited by 3,848
  12. 2021The Power of Scale for Parameter-Efficient Prompt Tuningcited by 3,769
38 more
  1. 2013Describing Textures in the Wildcited by 2,110
  2. 2020The Many Faces of Robustness: A Critical Analysis of Out-of-Distribution Generalizationcited by 1,842
  3. 2013Fine-Grained Visual Classification of Aircraftcited by 1,701
  4. 2019Theoretically Principled Trade-off between Robustness and Accuracycited by 1,510
  5. 2019Do ImageNet Classifiers Generalize to ImageNet?cited by 1,507
  6. 2017EuroSAT: A Novel Dataset and Deep Learning Benchmark for Land Use and Land Cover Classificationcited by 1,488
  7. 2019Natural Adversarial Examplescited by 1,401
  8. 2020WILDS: A Benchmark of in-the-Wild Distribution Shiftscited by 1,366
  9. 2021BitFit: Simple Parameter-efficient Fine-tuning for Transformer-based Masked Language-modelscited by 1,234
  10. 2021Grounded Language-Image Pre-trainingcited by 1,147
  11. 2019Learning Robust Global Representations by Penalizing Local Predictive Powercited by 1,010
  12. 2021StyleCLIP: Text-Driven Manipulation of StyleGAN Imagerycited by 990
  13. 2021Learning to Prompt for Continual Learningcited by 938
  14. 2019AugMix: A Simple Data Processing Method to Improve Robustness and Uncertaintycited by 897
  15. 2018Robustness May Be at Odds with Accuracycited by 873
  16. 2020Do We Really Need to Access the Source Data? Source Hypothesis Transfer for Unsupervised Domain Adaptationcited by 844
  17. 2021MDETR -- Modulated Detection for End-to-End Multi-Modal Understandingcited by 811
  18. 2021Robust fine-tuning of zero-shot modelscited by 785
  19. 2022Fine-Tuning can Distort Pretrained Features and Underperform Out-of-Distributioncited by 722
  20. 2021LiT: Zero-Shot Transfer with Locked-image text Tuningcited by 572
  21. 2016Regularization With Stochastic Transformations and Perturbations for Deep Semi-Supervised Learningcited by 441
  22. 2020Improving robustness against common corruptions by covariate shift adaptationcited by 427
  23. 2021MEMO: Test Time Robustness via Adaptation and Augmentationcited by 376
  24. 2019Semi-supervised Domain Adaptation via Minimax Entropycited by 333
  25. 2022Learning to Prompt for Open-Vocabulary Object Detection with Vision-Language Modelcited by 317
  26. 2016Revisiting Batch Normalization For Practical Domain Adaptationcited by 285
  27. 2017"Zero-Shot" Super-Resolution using Deep Internal Learningcited by 285
  28. 2020Universal Source-Free Domain Adaptationcited by 219
  29. 2020Semantic Photo Manipulation with a Generative Image Priorcited by 178
  30. 2018Reducing Network Agnostophobiacited by 165
  31. 2021End-to-End Human Object Interaction Detection with HOI Transformercited by 134
  32. 2022Prompt Consistency for Zero-Shot Task Generalizationcited by 66
  33. 2020Bongard-LOGO: A New Benchmark for Human-Level Concept Learning and Reasoningcited by 63
  34. 2022IDPG: An Instance-Dependent Prompt Generation Methodcited by 43
  35. 2022X-DETR: A Versatile Architecture for Instance-wise Vision-Language Taskscited by 41
  36. 2022Bongard-HOI: Benchmarking Few-Shot Visual Reasoning for Human-Object Interactionscited by 34
  37. 2022CLIP-TD: CLIP Targeted Distillation for Vision-Language Taskscited by 32
  38. 2020Neural Networks with Recurrent Generative Feedbackcited by 23

50 most cited of 53 All 53 (JSON)

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2022: 6; 2023: 67; 2024: 102; 2025: 121; 2026: 83.

379 papers.

Sort:

By area:

  1. 2023Vision-Language Models for Vision Tasks: A Surveycited by 834
  2. 2022MaPLe: Multi-modal Prompt Learningcited by 685
  3. 2024Parameter-Efficient Fine-Tuning for Large Models: A Comprehensive Surveycited by 641
  4. 2023A Comprehensive Survey on Test-Time Adaptation under Distribution Shiftscited by 353
  5. 2023Self-regulating Prompts: Foundational Model Adaptation without Forgettingcited by 272
  6. 2023Robust Test-Time Adaptation in Dynamic Scenarioscited by 185
  7. 2025The Unreasonable Effectiveness of Entropy Minimization in LLM Reasoningcited by 160
  8. 2023Diverse Data Augmentation with Diffusions for Effective Test-time Prompt Tuningcited by 150
  9. 2024Efficient Test-Time Adaptation of Vision-Language Modelscited by 138
  10. 2024A Comprehensive Survey of Small Language Models in the Era of Large Language Models: Techniques, Enhancements, Applications, Collaboration with LLMs, and Trustworthinesscited by 125
  11. 2022SuS-X: Training-Free Name-Only Transfer of Vision-Language Modelscited by 118
  12. 2022Understanding Zero-Shot Adversarial Robustness for Large-Scale Modelscited by 117
38 more
  1. 2024Low-Rank Few-Shot Adaptation of Vision-Language Modelscited by 103
  2. 2023Waffling around for Performance: Visual Classification with Random Words and Broad Conceptscited by 100
  3. 2023Align Your Prompts: Test-Time Prompting with Distribution Alignment for Zero-Shot Generalizationcited by 100
  4. 2023CLIPood: Generalizing CLIP to Out-of-Distributionscited by 96
  5. 2023Fine-tuning can cripple your foundation model; preserving features may be the solutioncited by 92
  6. 2023Not All Features Matter: Enhancing Few-shot CLIP with Adaptive Prior Refinementcited by 84
  7. 2024C-TPT: Calibrated Test-Time Prompt Tuning for Vision-Language Models via Text Feature Dispersioncited by 82
  8. 2023A Foundation Language-Image Model of the Retina (FLAIR): Encoding Expert Knowledge in Text Supervisioncited by 78
  9. 2024Test-Time Model Adaptation with Only Forward Passescited by 74
  10. 2024PromptKD: Unsupervised Prompt Distillation for Vision-Language Modelscited by 72
  11. 2023GraphAdapter: Tuning Vision-Language Models With Dual Knowledge Graphcited by 69
  12. 2024On the test-time zero-shot generalization of vision-language models: Do we really need prompt learning?cited by 65
  13. 2023BadCLIP: Trigger-Aware Prompt Learning for Backdoor Attacks on CLIPcited by 63
  14. 2023A Survey on Open-Vocabulary Detection and Segmentation: Past, Present, and Futurecited by 59
  15. 2024Improved Zero-Shot Classification by Adapting VLMs with Text Descriptionscited by 52
  16. 2022Bayesian Prompt Learning for Image-Language Model Generalizationcited by 51
  17. 2022Texts as Images in Prompt Tuning for Multi-Label Image Recognitioncited by 51
  18. 2024A Hard-to-Beat Baseline for Training-free CLIP-based Adaptationcited by 51
  19. 2024Dual Memory Networks: A Versatile Adaptation Approach for Vision-Language Modelscited by 51
  20. 2025Test-Time Learning for Large Language Modelscited by 46
  21. 2022Improving Zero-shot Generalization and Robustness of Multi-modal Modelscited by 44
  22. 2023LaFTer: Label-Free Tuning of Zero-shot Classifier using Language and Unlabeled Image Collectionscited by 43
  23. 2023In Search of Lost Online Test-time Adaptation: A Surveycited by 43
  24. 2025Parameter-Efficient Fine-Tuning for Foundation Modelscited by 43
  25. 2023ID-like Prompt Learning for Few-Shot Out-of-Distribution Detectioncited by 42
  26. 2023Leveraging Vision-Language Models for Improving Domain Generalization in Image Classificationcited by 41
  27. 2025Centaur: Robust End-to-End Autonomous Driving with Test-Time Trainingcited by 41
  28. 2023Continual Vision-Language Representation Learning with Off-Diagonal Informationcited by 40
  29. 2023A Simple Zero-shot Prompt Weighting Technique to Improve Prompt Ensembling in Text-Image Modelscited by 39
  30. 2023Black Box Few-Shot Adaptation for Vision-Language modelscited by 39
  31. 2024Few-Shot Adversarial Prompt Learning on Vision-Language Modelscited by 39
  32. 2023How Does Fine-Tuning Impact Out-of-Distribution Detection for Vision-Language Models?cited by 38
  33. 2023Test-Time Adaptation with CLIP Reward for Zero-Shot Generalization in Vision-Language Modelscited by 37
  34. 2024Learning to Prompt with Text Only Supervision for Vision-Language Modelscited by 37
  35. 2023StyLIP: Multi-Scale Style-Conditioned Prompt Learning for CLIP-based Domain Generalizationcited by 36
  36. 2023Language Models as Black-Box Optimizers for Vision-Language Modelscited by 36
  37. 2024Frustratingly Easy Test-Time Adaptation of Vision-Language Modelscited by 36
  38. 2024Boosting Vision-Language Models with Transductioncited by 36

50 most cited of 379 All 379 (JSON)

Similar, not cited

Close in wording, no citation either way

  1. 2024Adapting Vision-Language Models to Open Classes via Test-Time Prompt Tuning
  2. 2024Generalizable Prompt Tuning for Vision-Language Models
  3. 2025Doubly Debiased Test-Time Prompt Tuning for Vision-Language Models
  4. 2023Gradient-Regulated Meta-Prompt Learning for Generalizable Vision-Language Models
  5. 2025MINT: Memory-Infused Prompt Tuning at Test-time for CLIP
  6. 2025A Closer Look at Conditional Prompt Tuning for Vision-Language Models
  7. 2022Unified Vision and Language Prompt Learning
  8. 2024Improving Zero-shot Generalization of Learned Prompts via Unsupervised Knowledge Distillation