mlsanityData to 7 Oct 2026

Soft Prompt Generation for Domain Generalization

arXiv 2404.19286 ↗First posted 30 Apr 2024cs.CVcited by 24

On the map: Multimodal LLMs / vision-language models → Prompt learning and adaptation of CLIP

Abstract (arXiv)

Large pre-trained vision language models (VLMs) have shown impressive zero-shot ability on downstream tasks with manually designed prompt. To further adapt VLMs to downstream tasks, soft prompt is proposed to replace manually designed prompt, which undergoes fine-tuning based on specific domain data. Prior prompt learning methods primarily learn a fixed prompt or residuled prompt from training samples. However, the learned prompts lack diversity and ignore information about unseen domains. In this paper, we reframe the prompt learning framework from a generative perspective and propose a simple yet efficient method for the Domain Generalization (DG) task, namely Soft Prompt Generation (SPG). Specifically, SPG consists of a two-stage training phase and an inference phase. During the training phase, we introduce soft prompt label for each domain, aiming to incorporate the generative model domain knowledge. During the inference phase, the generator of the generative model is employed to obtain instance-specific soft prompts for the unseen target domain. Extensive experiments on five domain generalization benchmarks of three DG tasks demonstrate that SPG achieves state-of-the-art performance. The code is available at https://github.com/renytek13/Soft-Prompt-Generation-with-CGAN.

Key facts. Cited by 24 papers in the map (data through 7 Oct 2026). Placed in Prompt learning and adaptation of CLIP, one of 16 directions in Multimodal LLMs / vision-language models. Builds on 34 of its 40 references that are in the map. First posted on arXiv 30 Apr 2024.

Builds on

34 of 40 references.

Closest ancestors · 4

Same direction, newest first

  1. 2024A Sober Look at the Robustness of CLIPs to Spurious Features18 Mar
  2. 2023Prompt-based Distribution Alignment for Unsupervised Domain Adaptation15 Dec
  3. 2022Conditional Prompt Learning for Vision-Language Models10 Mar
  4. 2021Learning to Prompt for Vision-Language Models2 Sep

Other references · 30

  1. 2015Deep Residual Learning for Image Recognitioncited by 51,809
  2. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,456
  3. 2020An Image is Worth 16x16 Words: Transformers for Image Recognition at Scalecited by 29,915
  4. 2021Scaling Up Visual and Vision-Language Representation Learning With Noisy Text Supervisioncited by 3,848
  5. 2015Domain-Adversarial Training of Neural Networkscited by 3,023
  6. 2014Conditional Generative Adversarial Netscited by 2,152
  7. 2022Visual Prompt Tuningcited by 1,674
  8. 2018Moment Matching for Multi-Source Domain Adaptationcited by 1,327
  9. 2017Deep Hashing Network for Unsupervised Domain Adaptationcited by 1,181
  10. 2020In Search of Lost Domain Generalizationcited by 1,042
  11. 2017Deeper, Broader and Artier Domain Generalizationcited by 983
  12. 2017Learning to Generalize: Meta-Learning for Domain Generalizationcited by 762
18 more
  1. 2021Domain Generalization: A Surveycited by 760
  2. 2022MaPLe: Multi-modal Prompt Learningcited by 685
  3. 2018Recognition in Terra Incognitacited by 655
  4. 2013Domain Generalization via Invariant Feature Representationcited by 622
  5. 2018Generalizing to Unseen Domains via Adversarial Data Augmentationcited by 451
  6. 2021SWAD: Domain Generalization by Seeking Flat Minimacited by 431
  7. 2013DeCAF: A Deep Convolutional Activation Feature for Generic Visual Recognitioncited by 422
  8. 2021Learning How to Ask: Querying LMs with Mixtures of Soft Promptscited by 376
  9. 2022Exploring Visual Prompts for Adapting Large-Scale Modelscited by 271
  10. 2022Domain Generalization by Mutual-Information Regularization with Pre-trained Modelscited by 133
  11. 2022Domain Adaptation via Prompt Learningcited by 128
  12. 2020Domain Generalization for Medical Imaging Classification with Linear-Dependency Regularizationcited by 122
  13. 2023PromptStyler: Prompt-driven Style Generation for Source-free Domain Generalizationcited by 59
  14. 2022Prompt Vision Transformer for Domain Generalizationcited by 52
  15. 2022Bayesian Prompt Learning for Image-Language Model Generalizationcited by 51
  16. 2021Domain Prompt Learning for Efficiently Adapting CLIP to Unseen Domainscited by 39
  17. 2022Domain-Unified Prompt Representations for Source-Free Domain Generalizationcited by 21
  18. 2023Dialogue for Prompting: a Policy-Gradient-Based Discrete Prompt Generation for Few-shot Learningcited by 7

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2024: 2; 2025: 17; 2026: 5.

24 papers.

Sort:

By area:

  1. 2025Advances in Multimodal Adaptation and Generalization: From Traditional Approaches to Foundation Modelscited by 23
  2. 2025Prompt-based Adaptation in Large-scale Vision Models: A Surveycited by 21
  3. 2025CLIP-Powered Domain Generalization and Domain Adaptation: A Comprehensive Surveycited by 15
  4. 2025Handling Out-of-Distribution Data: A Surveycited by 14
  5. 2025Prompt Disentanglement via Language Guidance and Representation Alignment for Domain Generalizationcited by 8
  6. 2026Reasoning-Driven Multimodal LLM for Domain Generalizationcited by 7
  7. 2024PromptTA: Prompt-driven Text Adapter for Source-free Domain Generalizationcited by 6
  8. 2025Generalizing vision-language models to novel domains: A comprehensive surveycited by 6
  9. 2025Exploiting Domain Properties in Language-Driven Domain Generalization for Semantic Segmentationcited by 4
  10. 2025Harmonizing and Merging Source Models for CLIP-based Domain Generalizationcited by 2
  11. 2025Simulate, Refocus and Ensemble: An Attention-Refocusing Scheme for Domain Generalizationcited by 2
  12. 2025Federated Domain Generalization with Domain-specific Soft Prompts Generationcited by 2
12 more
  1. 2024Attention Head Purification: A New Perspective to Harness CLIP for Domain Generalizationcited by 1
  2. 2025A Language Vision Model Approach for Automated Tumor Contouring in Radiation Oncologycited by 1
  3. 2025Text-Driven Causal Representation Learning for Source-Free Domain Generalizationcited by 1
  4. 2025Latent Domain Prompt Learning for Vision-Language Modelscited by 1
  5. 2025Generalizing Vision-Language Models with Dedicated Prompt Guidancecited by 1
  6. 2025Dual-Path Stable Soft Prompt Generation for Domain Generalizationcited by 0
  7. 2025Preserving Domain Generalization in Fine-Tuning via Joint Parameter Selectioncited by 0
  8. 2025Domain Generalization in-the-Wild: Disentangling Classification from Domain-Aware Representationscited by 0
  9. 2026OmniVaT: Single Domain Generalization for Multimodal Visual-Tactile Learningcited by 0
  10. 2026SemAlign: Language Guided Semi-supervised Domain Generalizationcited by 0
  11. 2026Optimizing Soft Prompt Tuning via Structural Evolutioncited by 0
  12. 2026Hierarchical Prompt Injector for Domain Generalization Segmentationcited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2024Transitive Vision-Language Prompt Learning for Domain Generalization
  2. 2022Learning Domain Invariant Prompt for Vision-Language Models
  3. 2024Improving Zero-shot Generalization of Learned Prompts via Unsupervised Knowledge Distillation
  4. 2024Generalizable Prompt Tuning for Vision-Language Models
  5. 2026Domain-Invariant Prompt Learning for Vision-Language Models
  6. 2022Test-Time Prompt Tuning for Zero-Shot Generalization in Vision-Language Models
  7. 2023Gradient-Regulated Meta-Prompt Learning for Generalizable Vision-Language Models
  8. 2026Your Model Already Knows Don't Teach It, Learn to Ask It: Soft Prompting for Few-Shot Adaptation of Vision-Language Models