mlsanityData to 7 Oct 2026

Align Your Prompts: Test-Time Prompting with Distribution Alignment for Zero-Shot Generalization

arXiv 2311.01459 ↗First posted 2 Nov 2023cs.CVcited by 100

On the map: Multimodal LLMs / vision-language models → Prompt learning and adaptation of CLIP

Abstract (arXiv)

The promising zero-shot generalization of vision-language models such as CLIP has led to their adoption using prompt learning for numerous downstream tasks. Previous works have shown test-time prompt tuning using entropy minimization to adapt text prompts for unseen domains. While effective, this overlooks the key cause for performance degradation to unseen domains -- distribution shift. In this work, we explicitly handle this problem by aligning the out-of-distribution (OOD) test sample statistics to those of the source data using prompt tuning. We use a single test sample to adapt multi-modal prompts at test time by minimizing the feature distribution shift to bridge the gap in the test domain. Evaluating against the domain generalization benchmark, our method improves zero-shot top- 1 accuracy beyond existing prompt-learning techniques, with a 3.08% improvement over the baseline MaPLe. In cross-dataset generalization with unseen categories across 10 datasets, our method improves consistently across all datasets compared to the existing state-of-the-art. Our source code and models are available at https://jameelhassan.github.io/promptalign.

Key facts. Cited by 100 papers in the map (data through 7 Oct 2026). Placed in Prompt learning and adaptation of CLIP, one of 16 directions in Multimodal LLMs / vision-language models. Builds on 35 of its 48 references that are in the map. First posted on arXiv 2 Nov 2023.

Builds on

35 of 48 references.

Closest ancestors · 4

Same direction, newest first

  1. 2022Unified Vision and Language Prompt Learning13 Oct
  2. 2022Test-Time Prompt Tuning for Zero-Shot Generalization in Vision-Language Models15 Sep
  3. 2022Conditional Prompt Learning for Vision-Language Models10 Mar
  4. 2021Learning to Prompt for Vision-Language Models2 Sep

Other references · 31

  1. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,456
  2. 2021Scaling Up Visual and Vision-Language Representation Learning With Noisy Text Supervisioncited by 3,848
  3. 2015Delving Deep into Rectifiers: Surpassing Human-Level Performance on ImageNet Classificationcited by 3,590
  4. 2019Benchmarking Neural Network Robustness to Common Corruptions and Perturbationscited by 2,857
  5. 2013Describing Textures in the Wildcited by 2,110
  6. 2020The Many Faces of Robustness: A Critical Analysis of Out-of-Distribution Generalizationcited by 1,842
  7. 2013Fine-Grained Visual Classification of Aircraftcited by 1,701
  8. 2021LAION-400M: Open Dataset of CLIP-Filtered 400 Million Image-Text Pairscited by 1,544
  9. 2019Do ImageNet Classifiers Generalize to ImageNet?cited by 1,507
  10. 2017EuroSAT: A Novel Dataset and Deep Learning Benchmark for Land Use and Land Cover Classificationcited by 1,488
  11. 2019Natural Adversarial Examplescited by 1,401
  12. 2022Reproducible scaling laws for contrastive language-image learningcited by 1,163
19 more
  1. 2019Learning Robust Global Representations by Penalizing Local Predictive Powercited by 1,010
  2. 2021Florence: A New Foundation Model for Computer Visioncited by 802
  3. 2022Fine-Tuning can Distort Pretrained Features and Underperform Out-of-Distributioncited by 722
  4. 2022MaPLe: Multi-modal Prompt Learningcited by 685
  5. 2022EVA: Exploring the Limits of Masked Visual Representation Learning at Scalecited by 685
  6. 2021FILIP: Fine-grained Interactive Language-Image Pre-Trainingcited by 626
  7. 2021LiT: Zero-Shot Transfer with Locked-image text Tuningcited by 572
  8. 2022Continual Test-Time Domain Adaptationcited by 520
  9. 2020Improving robustness against common corruptions by covariate shift adaptationcited by 427
  10. 2021MEMO: Test Time Robustness via Adaptation and Augmentationcited by 376
  11. 2023Self-regulating Prompts: Foundational Model Adaptation without Forgettingcited by 272
  12. 2022Exploring Visual Prompts for Adapting Large-Scale Modelscited by 271
  13. 2021Prototypical Pseudo Label Denoising and Target Structure Learning for Domain Adaptive Semantic Segmentationcited by 268
  14. 2022Test-Time Training with Masked Autoencoderscited by 208
  15. 2017Central Moment Discrepancy (CMD) for Domain-Invariant Representation Learningcited by 192
  16. 2021The Norm Must Go On: Dynamic Unsupervised Domain Adaptation by Normalizationcited by 131
  17. 2022ActMAD: Activation Matching to Align Distributions for Test-Time-Trainingcited by 38
  18. 2022Video Test-Time Adaptation for Action Recognitioncited by 35
  19. 2023PUG: Photorealistic and Semantically Controllable Synthetic Data for Representation Learningcited by 31

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2023: 1; 2024: 35; 2025: 45; 2026: 19.

100 papers.

Sort:

By area:

  1. 2024C-TPT: Calibrated Test-Time Prompt Tuning for Vision-Language Models via Text Feature Dispersioncited by 82
  2. 2024PromptKD: Unsupervised Prompt Distillation for Vision-Language Modelscited by 72
  3. 2025A Survey of Test-Time Compute: From Intuitive Inference to Deliberate Reasoningcited by 40
  4. 2024Learning to Prompt with Text Only Supervision for Vision-Language Modelscited by 37
  5. 2024Frustratingly Easy Test-Time Adaptation of Vision-Language Modelscited by 36
  6. 2024Boosting Vision-Language Models with Transductioncited by 36
  7. 2024Dual Prototype Evolving for Test-Time Generalization of Vision-Language Modelscited by 36
  8. 2025R-TPT: Improving Adversarial Robustness of Vision-Language Models through Test-Time Prompt Tuningcited by 33
  9. 2024Just Shift It: Test-Time Prototype Shifting for Zero-Shot Generalization with Vision-Language Modelscited by 31
  10. 2024AWT: Transferring Vision-Language Models via Augmentation, Weighting, and Transportationcited by 27
  11. 2024Enhancing Zero-Shot Vision Models by Label-Free Prompt Distribution Learning and Bias Correctingcited by 26
  12. 2024Any-Shift Prompting for Generalization over Distributionscited by 24
38 more
  1. 2025Advances in Multimodal Adaptation and Generalization: From Traditional Approaches to Foundation Modelscited by 23
  2. 2024TAPT: Test-Time Adversarial Prompt Tuning for Robust Inference in Vision-Language Modelscited by 22
  3. 2024Historical Test-time Prompt Tuning for Vision Foundation Modelscited by 21
  4. 2025Bayesian Test-Time Adaptation for Vision-Language Modelscited by 21
  5. 2025Prompt-based Adaptation in Large-scale Vision Models: A Surveycited by 21
  6. 2024Test-Time Zero-Shot Temporal Action Localizationcited by 19
  7. 2024Improving Zero-shot Generalization of Learned Prompts via Unsupervised Knowledge Distillationcited by 18
  8. 2024Progressive Semantic-Guided Vision Transformer for Zero-Shot Learningcited by 17
  9. 2024OT-VP: Optimal Transport-guided Visual Prompting for Test-Time Adaptationcited by 17
  10. 2024Test-Time Low Rank Adaptation via Confidence Maximization for Zero-Shot Generalization of Vision-Language Modelscited by 17
  11. 2024BoostAdapter: Improving Vision-Language Test-Time Adaptation via Regional Bootstrappingcited by 17
  12. 2024Visual-Augmented Dynamic Semantic Prototype for Generative Zero-Shot Learningcited by 16
  13. 2025ProKeR: A Kernel Perspective on Few-Shot Adaptation of Large Vision-Language Modelscited by 15
  14. 2025COSMIC: Clique-Oriented Semantic Multi-space Integration for Robust CLIP Test-Time Adaptationcited by 13
  15. 2025ProAPO: Progressively Automatic Prompt Optimization for Visual Classificationcited by 12
  16. 2024Efficient Test-Time Prompt Tuning for Vision-Language Modelscited by 9
  17. 2025Latte: Collaborative Test-Time Adaptation of Vision-Language Models in Federated Learningcited by 9
  18. 2024Improving Black-box Robustness with In-Context Rewritingcited by 8
  19. 2025Noisy Test-Time Adaptation in Vision-Language Modelscited by 8
  20. 2025ICPL-ReID: Identity-Conditional Prompt Learning for Multi-Spectral Object Re-Identificationcited by 8
  21. 2025Free on the Fly: Enhancing Flexibility in Test-Time Adaptation with Online EMcited by 8
  22. 2023Descriptor and Word Soups: Overcoming the Parameter Efficiency Accuracy Tradeoff for Out-of-Distribution Few-shot Learningcited by 6
  23. 2024Out-of-Distribution Learning with Human Feedbackcited by 6
  24. 2025Confidence-calibrated covariate shift correction for few-shot classification in Vision-Language Modelscited by 6
  25. 2025Generalizing vision-language models to novel domains: A comprehensive surveycited by 6
  26. 2025Multi-Cache Enhanced Prototype Learning for Test-Time Generalization of Vision-Language Modelscited by 6
  27. 2024Unified Source-Free Domain Adaptationcited by 5
  28. 2024On the Element-Wise Representation and Reasoning in Zero-Shot Image Recognition: A Systematic Surveycited by 5
  29. 2024Is Less More? Exploring Token Condensation as Training-free Test-time Adaptationcited by 5
  30. 2025Evaluating and Improving Robustness in Large Language Models: A Survey and Future Directionscited by 5
  31. 2025Backpropagation-Free Test-Time Adaptation via Probabilistic Gaussian Alignmentcited by 5
  32. 2025Mint: A Simple Test-Time Adaptation of Vision-Language Models against Common Corruptionscited by 5
  33. 2026Test-Time Adaptation via Cache Personalization for Facial Expression Recognition in Videoscited by 5
  34. 2024Bootstrap Segmentation Foundation Model under Distribution Shift via Object-Centric Learningcited by 4
  35. 2024Efficient Open Set Single Image Test Time Adaptation of Vision Language Modelscited by 3
  36. 2025Test-Time Optimization for Domain Adaptive Open Vocabulary Segmentationcited by 3
  37. 2025Diff-Prompt: Diffusion-Driven Prompt Generator with Mask Supervisioncited by 3
  38. 2025ETTA: Efficient Test-Time Adaptation for Vision-Language Models through Dynamic Embedding Updatescited by 3

50 most cited of 100 All 100 (JSON)

Similar, not cited

Close in wording, no citation either way

  1. 2024Generalizable Prompt Tuning for Vision-Language Models
  2. 2024Adapting Vision-Language Models to Open Classes via Test-Time Prompt Tuning
  3. 2025Doubly Debiased Test-Time Prompt Tuning for Vision-Language Models
  4. 2023Understanding prompt engineering may not require rethinking generalization
  5. 2022Learning Domain Invariant Prompt for Vision-Language Models
  6. 2025Dual-Path Stable Soft Prompt Generation for Domain Generalization
  7. 2024On the test-time zero-shot generalization of vision-language models: Do we really need prompt learning?
  8. 2025Learning Generalizable Prompt for CLIP with Class Similarity Knowledge