mlsanityData to 7 Oct 2026

Learning Domain Invariant Prompt for Vision-Language Models

arXiv 2212.04196 ↗First posted 8 Dec 2022cs.CVcited by 35

On the map: Multimodal LLMs / vision-language models → Prompt learning and adaptation of CLIP

Abstract (arXiv)

Prompt learning is one of the most effective and trending ways to adapt powerful vision-language foundation models like CLIP to downstream datasets by tuning learnable prompt vectors with very few samples. However, although prompt learning achieves excellent performance over in-domain data, it still faces the major challenge of generalizing to unseen classes and domains. Some existing prompt learning methods tackle this issue by adaptively generating different prompts for different tokens or domains but neglecting the ability of learned prompts to generalize to unseen domains. In this paper, we propose a novel prompt learning paradigm that directly generates domain invariant prompt that can be generalized to unseen domains, called MetaPrompt. Specifically, a dual-modality prompt tuning network is proposed to generate prompts for input from both image and text modalities. With a novel asymmetric contrastive loss, the representation from the original pre-trained vision-language model acts as supervision to enhance the generalization ability of the learned prompt. More importantly, we propose a meta-learning-based prompt tuning algorithm that explicitly constrains the task-specific prompt tuned for one domain or class to also achieve good performance in another domain or class. Extensive experiments on 11 datasets for base-to-new generalization and 4 datasets for domain generalization demonstrate that our method consistently and significantly outperforms existing methods.

Key facts. Cited by 35 papers in the map (data through 7 Oct 2026). Placed in Prompt learning and adaptation of CLIP, one of 16 directions in Multimodal LLMs / vision-language models. Builds on 57 of its 96 references that are in the map. First posted on arXiv 8 Dec 2022.

Builds on

57 of 96 references.

Closest ancestors · 4

Same direction, newest first

  1. 2022Dual Modality Prompt Tuning for Vision-Language Pre-Trained Model17 Aug
  2. 2022Prompt Distribution Learning6 May
  3. 2022Conditional Prompt Learning for Vision-Language Models10 Mar
  4. 2021Learning to Prompt for Vision-Language Models2 Sep

Other references · 53

  1. 2017Attention Is All You Needcited by 59,932
  2. 2015Deep Residual Learning for Image Recognitioncited by 51,809
  3. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,204
  4. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,456
  5. 2020An Image is Worth 16x16 Words: Transformers for Image Recognition at Scalecited by 29,915
  6. 2017Model-Agnostic Meta-Learning for Fast Adaptation of Deep Networkscited by 4,861
  7. 2021Prefix-Tuning: Optimizing Continuous Prompts for Generationcited by 3,878
  8. 2021Scaling Up Visual and Vision-Language Representation Learning With Noisy Text Supervisioncited by 3,848
  9. 2021The Power of Scale for Parameter-Efficient Prompt Tuningcited by 3,769
  10. 2017Prototypical Networks for Few-shot Learningcited by 3,001
  11. 2016Matching Networks for One Shot Learningcited by 2,246
  12. 2021Pre-train, Prompt, and Predict: A Systematic Survey of Prompting Methods in Natural Language Processingcited by 2,222
38 more
  1. 2013Describing Textures in the Wildcited by 2,110
  2. 2014Unsupervised Domain Adaptation by Backpropagationcited by 1,906
  3. 2019Invariant Risk Minimizationcited by 1,832
  4. 2013Fine-Grained Visual Classification of Aircraftcited by 1,701
  5. 2022Visual Prompt Tuningcited by 1,674
  6. 2017EuroSAT: A Novel Dataset and Deep Learning Benchmark for Land Use and Land Cover Classificationcited by 1,488
  7. 2018Moment Matching for Multi-Source Domain Adaptationcited by 1,327
  8. 2016Deep CORAL: Correlation Alignment for Deep Domain Adaptationcited by 1,194
  9. 2017Deep Hashing Network for Unsupervised Domain Adaptationcited by 1,181
  10. 2017Learning to Compare: Relation Network for Few-Shot Learningcited by 1,139
  11. 2020In Search of Lost Domain Generalizationcited by 1,042
  12. 2018On First-Order Meta-Learning Algorithmscited by 1,003
  13. 2017Deeper, Broader and Artier Domain Generalizationcited by 983
  14. 2017Learning to Generalize: Meta-Learning for Domain Generalizationcited by 762
  15. 2021Domain Generalization: A Surveycited by 760
  16. 2021P-Tuning v2: Prompt Tuning Can Be Comparable to Fine-tuning Universally Across Scales and Taskscited by 686
  17. 2022MaPLe: Multi-modal Prompt Learningcited by 685
  18. 2018Disentangling by Factorisingcited by 614
  19. 2019Meta-Learning with Implicit Gradientscited by 523
  20. 2019Meta-Learning with Differentiable Convex Optimizationcited by 461
  21. 2021SWAD: Domain Generalization by Seeking Flat Minimacited by 431
  22. 2018TADAM: Task dependent adaptive metric for improved few-shot learningcited by 408
  23. 2018Meta-learning with differentiable closed-form solverscited by 395
  24. 2019Rapid Learning or Feature Reuse? Towards Understanding the Effectiveness of MAMLcited by 359
  25. 2017Unified Deep Supervised Domain Adaptation and Generalizationcited by 322
  26. 2018How to train your MAMLcited by 306
  27. 2017Meta Networkscited by 268
  28. 2021Gradient Matching for Domain Generalizationcited by 258
  29. 2017Feature Generating Networks for Zero-Shot Learningcited by 257
  30. 2017Zero-Shot Learning -- The Good, the Bad and the Uglycited by 254
  31. 2018Probabilistic Model-Agnostic Meta-Learningcited by 254
  32. 2018Recasting Gradient-Based Meta-Learning as Hierarchical Bayescited by 180
  33. 2021A Fine-Grained Analysis on Distribution Shiftcited by 171
  34. 2018Zero-shot Recognition via Semantic Embeddings and Knowledge Graphscited by 155
  35. 2016An Empirical Study and Analysis of Generalized Zero-Shot Learning for Object Recognition in the Wildcited by 146
  36. 2021Ensemble of Averages: Improving Model Selection and Boosting Performance in Domain Generalizationcited by 144
  37. 2019Meta-Learning with Warped Gradient Descentcited by 98
  38. 2018Rethinking Knowledge Graph Propagation for Zero-Shot Learningcited by 87

50 most cited of 53 All 53 (JSON)

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2023: 3; 2024: 10; 2025: 16; 2026: 6.

35 papers.

Sort:

By area:

  1. 2023Visual Tuningcited by 34
  2. 2023RefSAM: Efficiently Adapting Segmenting Anything Model for Referring Video Object Segmentationcited by 24
  3. 2024MSCPT: Few-shot Whole Slide Image Classification with Multi-scale and Context-focused Prompt Tuningcited by 16
  4. 2024Enhancing Perception of Key Changes in Remote Sensing Image Change Captioningcited by 16
  5. 2026Explore with Long-term Memory: A Benchmark and Multimodal LLM-based Reinforcement Learning Framework for Embodied Explorationcited by 13
  6. 2025Hierarchical Cross-modal Prompt Learning for Vision-Language Modelscited by 12
  7. 2025IAP: Improving Continual Learning of Vision-Language Models via Instance-Aware Promptingcited by 7
  8. 2024Adversarial Prompt Distillation for Vision-Language Modelscited by 6
  9. 2025CKAA: Cross-subspace Knowledge Alignment and Aggregation for Robust Continual Learningcited by 5
  10. 2026MMLoP: Multi-Modal Low-Rank Prompting for Efficient Vision-Language Adaptationcited by 5
  11. 2024Multi-Stage Knowledge Integration of Vision-Language Models for Continual Learningcited by 4
  12. 2025MMRL++: Parameter-Efficient and Interaction-Aware Representation Learning for Vision-Language Modelscited by 4
23 more
  1. 2025UPRE: Zero-Shot Domain Adaptation for Object Detection via Unified Prompt and Representation Enhancementcited by 4
  2. 2026Adversarial Domain Prompt Tuning and Generation for Single Domain Generalizationcited by 4
  3. 2024SETA: Semantic-Aware Token Augmentation for Domain Generalizationcited by 3
  4. 2024Empowering Source-Free Domain Adaptation via MLLM-Guided Reliability-Based Curriculum Learningcited by 3
  5. 2025MGPATH: Vision-Language Model with Multi-Granular Prompt Learning for Few-Shot WSI Classificationcited by 3
  6. 2024ActPrompt: In-Domain Feature Adaptation via Action Cues for Video Temporal Groundingcited by 2
  7. 2024HPT++: Hierarchically Prompting Vision-Language Models with Multi-Granularity Knowledge Generation and Improved Structure Modelingcited by 2
  8. 2025Exploring Interpretability for Visual Prompt Tuning with Cross-layer Conceptscited by 2
  9. 2023Can Out-of-Domain data help to Learn Domain-Specific Prompts for Multimodal Misinformation Detection?cited by 1
  10. 2024MoPD: Mixture-of-Prompts Distillation for Vision-Language Modelscited by 1
  11. 2025Simple yet Effective Semi-supervised Knowledge Distillation from Vision-Language Models via Dual-Head Optimizationcited by 1
  12. 2025Alternating Training-based Label Smoothing Enhances Prompt Generalizationcited by 1
  13. 2024Hierarchical Multi-modal Transformer for Cross-modal Long Document Classificationcited by 0
  14. 2025PCoreSet: Effective Active Learning through Knowledge Distillation from Vision-Language Modelscited by 0
  15. 2025Integrated Structural Prompt Learning for Vision-Language Modelscited by 0
  16. 2025Remodeling Semantic Relationships in Vision-Language Fine-Tuningcited by 0
  17. 2025Automatic Attack Discovery for Few-Shot Class-Incremental Learning via Large Language Modelscited by 0
  18. 2025Fourier-Attentive Representation Learning: A Fourier-Guided Framework for Few-Shot Generalization in Vision-Language Modelscited by 0
  19. 2025Pretrained battery transformer (PBT): A foundation model for battery life predictioncited by 0
  20. 2025AmPLe: Supporting Vision-Language Models via Adaptive-Debiased Ensemble Multi-Prompt Learningcited by 0
  21. 2026Guided Prompt Evolution for Vision-Language Models Adaptationcited by 0
  22. 2026World2Minecraft: Occupancy-Driven Simulated Scenes Constructioncited by 0
  23. 2026CMAP: Cross-Modal Adaptive Prompting for Multi-Domain Task-Incremental Learningcited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2023Gradient-Regulated Meta-Prompt Learning for Generalizable Vision-Language Models
  2. 2024Generalizable Prompt Tuning for Vision-Language Models
  3. 2022Unified Vision and Language Prompt Learning
  4. 2024Transitive Vision-Language Prompt Learning for Domain Generalization
  5. 2024PromptSync: Bridging Domain Gaps in Vision-Language Models through Class-Aware Prototype Alignment and Discrimination
  6. 2024Domain-Agnostic Mutual Prompting for Unsupervised Domain Adaptation
  7. 2025MetaTPT: Meta Test-time Prompt Tuning for Vision-Language Models
  8. 2024PromptKD: Unsupervised Prompt Distillation for Vision-Language Models