# Domain Prompt Learning for Efficiently Adapting CLIP to Unseen Domains

arXiv: https://arxiv.org/abs/2111.12853 · HTML: https://mlsanity.com/paper/2111.12853/

## Key facts

- Cited by 39 papers in the map (data through 14 Oct 2026).
- Builds on 34 of its 61 references that are in the map.
- First posted on arXiv 25 Nov 2021.

## Builds on (most cited first, 30 of 34)

- [Learning Transferable Visual Models From Natural Language Supervision](https://mlsanity.com/paper/2103.00020/) (2021)
- [An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale](https://mlsanity.com/paper/2010.11929/) (2020)
- [Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformer](https://mlsanity.com/paper/1910.10683/) (2019)
- [Training data-efficient image transformers & distillation through attention](https://mlsanity.com/paper/2012.12877/) (2020)
- [On the Opportunities and Risks of Foundation Models](https://mlsanity.com/paper/2108.07258/) (2021)
- [Prefix-Tuning: Optimizing Continuous Prompts for Generation](https://mlsanity.com/paper/2101.00190/) (2021)
- [The Power of Scale for Parameter-Efficient Prompt Tuning](https://mlsanity.com/paper/2104.08691/) (2021)
- [Domain-Adversarial Training of Neural Networks](https://mlsanity.com/paper/1505.07818/) (2015)
- [Learning to Prompt for Vision-Language Models](https://mlsanity.com/paper/2109.01134/) (2021)
- [Pre-train, Prompt, and Predict: A Systematic Survey of Prompting Methods in Natural Language Processing](https://mlsanity.com/paper/2107.13586/) (2021)
- [Unsupervised Domain Adaptation by Backpropagation](https://mlsanity.com/paper/1409.7495/) (2014)
- [Invariant Risk Minimization](https://mlsanity.com/paper/1907.02893/) (2019)
- [MLP-Mixer: An all-MLP Architecture for Vision](https://mlsanity.com/paper/2105.01601/) (2021)
- [WILDS: A Benchmark of in-the-Wild Distribution Shifts](https://mlsanity.com/paper/2012.07421/) (2020)
- [Deep CORAL: Correlation Alignment for Deep Domain Adaptation](https://mlsanity.com/paper/1607.01719/) (2016)
- [Deep Hashing Network for Unsupervised Domain Adaptation](https://mlsanity.com/paper/1706.07522/) (2017)
- [In Search of Lost Domain Generalization](https://mlsanity.com/paper/2007.01434/) (2020)
- [CLIP-Adapter: Better Vision-Language Models with Feature Adapters](https://mlsanity.com/paper/2110.04544/) (2021)
- [Deeper, Broader and Artier Domain Generalization](https://mlsanity.com/paper/1710.03077/) (2017)
- [Do We Really Need to Access the Source Data? Source Hypothesis Transfer for Unsupervised Domain Adaptation](https://mlsanity.com/paper/2002.08546/) (2020)
- [Big Transfer (BiT): General Visual Representation Learning](https://mlsanity.com/paper/1912.11370/) (2019)
- [Generalizing to Unseen Domains: A Survey on Domain Generalization](https://mlsanity.com/paper/2103.03097/) (2021)
- [Learning to Generalize: Meta-Learning for Domain Generalization](https://mlsanity.com/paper/1710.03463/) (2017)
- [P-Tuning v2: Prompt Tuning Can Be Comparable to Fine-tuning Universally Across Scales and Tasks](https://mlsanity.com/paper/2110.07602/) (2021)
- [Recognition in Terra Incognita](https://mlsanity.com/paper/1807.04975/) (2018)
- [Towards Out-Of-Distribution Generalization: A Survey](https://mlsanity.com/paper/2108.13624/) (2021)
- [SWAD: Domain Generalization by Seeking Flat Minima](https://mlsanity.com/paper/2102.08604/) (2021)
- [Domain Generalization via Model-Agnostic Learning of Semantic Features](https://mlsanity.com/paper/1910.13580/) (2019)
- [Unified Deep Supervised Domain Adaptation and Generalization](https://mlsanity.com/paper/1709.10190/) (2017)
- [Generalizing Across Domains via Cross-Gradient Training](https://mlsanity.com/paper/1804.10745/) (2018)

## Built on by (most cited first, 30 of 39)

- [CLIPood: Generalizing CLIP to Out-of-Distributions](https://mlsanity.com/paper/2302.00864/) (2023)
- [Improved Zero-Shot Classification by Adapting VLMs with Text Descriptions](https://mlsanity.com/paper/2401.02460/) (2024)
- [A Sentence Speaks a Thousand Images: Domain Generalization through Distilling CLIP with Language Guidance](https://mlsanity.com/paper/2309.12530/) (2023)
- [Domain-Agnostic Mutual Prompting for Unsupervised Domain Adaptation](https://mlsanity.com/paper/2403.02899/) (2024)
- [Enhancing CLIP with CLIP: Exploring Pseudolabeling for Limited-Label Prompt Tuning](https://mlsanity.com/paper/2306.01669/) (2023)
- [Context-Aware Robust Fine-Tuning](https://mlsanity.com/paper/2211.16175/) (2022)
- [Any-Shift Prompting for Generalization over Distributions](https://mlsanity.com/paper/2402.10099/) (2024)
- [Soft Prompt Generation for Domain Generalization](https://mlsanity.com/paper/2404.19286/) (2024)
- [Adapting to Distribution Shift by Visual Domain Prompt Generation](https://mlsanity.com/paper/2405.02797/) (2024)
- [Advances in Multimodal Adaptation and Generalization: From Traditional Approaches to Foundation Models](https://mlsanity.com/paper/2501.18592/) (2025)
- [CLIP-Powered Domain Generalization and Domain Adaptation: A Comprehensive Survey](https://mlsanity.com/paper/2504.14280/) (2025)
- [Learning to Adapt Frozen CLIP for Few-Shot Test-Time Domain Adaptation](https://mlsanity.com/paper/2506.17307/) (2025)
- [LanDA: Language-Guided Multi-Source Domain Adaptation](https://mlsanity.com/paper/2401.14148/) (2024)
- [Video Unsupervised Domain Adaptation with Deep Learning: A Comprehensive Survey](https://mlsanity.com/paper/2211.10412/) (2022)
- [Uncovering Adversarial Risks of Test-Time Adaptation](https://mlsanity.com/paper/2301.12576/) (2023)
- [Exploring the Transfer Learning Capabilities of CLIP in Domain Generalization for Diabetic Retinopathy](https://mlsanity.com/paper/2308.14212/) (2023)
- [Prompt Disentanglement via Language Guidance and Representation Alignment for Domain Generalization](https://mlsanity.com/paper/2507.02288/) (2025)
- [Generalizing vision-language models to novel domains: A comprehensive survey](https://mlsanity.com/paper/2506.18504/) (2025)
- [Enhanced Meta Label Correction for Coping with Label Corruption](https://mlsanity.com/paper/2305.12961/) (2023)
- [BDC-Adapter: Brownian Distance Covariance for Better Vision-Language Reasoning](https://mlsanity.com/paper/2309.01256/) (2023)
- [Towards Unified and Effective Domain Generalization](https://mlsanity.com/paper/2310.10008/) (2023)
- [Exploiting Domain Properties in Language-Driven Domain Generalization for Semantic Segmentation](https://mlsanity.com/paper/2512.03508/) (2025)
- [Tell, Don't Show!: Language Guidance Eases Transfer Across Domains in Images and Videos](https://mlsanity.com/paper/2403.05535/) (2024)
- [TDG: Text-guided Domain Generalization](https://mlsanity.com/paper/2308.09931/) (2023)
- [Learning Invariant Causal Mechanism from Vision-Language Models](https://mlsanity.com/paper/2405.15289/) (2024)
- [Federated Domain Generalization with Domain-specific Soft Prompts Generation](https://mlsanity.com/paper/2509.20807/) (2025)
- [VLLaVO: Mitigating Visual Gap through LLMs](https://mlsanity.com/paper/2401.03253/) (2024)
- [Transitive Vision-Language Prompt Learning for Domain Generalization](https://mlsanity.com/paper/2404.18758/) (2024)
- [Generate, Transduct, Adapt: Iterative Transduction with VLMs](https://mlsanity.com/paper/2501.06031/) (2025)
- [Enhancing Vision-Language Models Generalization via Diversity-Driven Novel Feature Synthesis](https://mlsanity.com/paper/2405.02586/) (2024)

## Abstract (arXiv)

Domain generalization (DG) is a difficult transfer learning problem aiming to learn a generalizable model for unseen domains. Recent foundation models (FMs) are robust to many distribution shifts and, therefore, should substantially improve the performance of DG. In this work, we study generic ways to adopt CLIP, a Visual-Language Foundation Model, for DG problems in image classification. While ERM greatly improves the accuracy with bigger backbones and training datasets using standard DG benchmarks, fine-tuning FMs is not practical in many real-world situations. We propose Domain Prompt Learning (DPL) as a novel approach for domain inference in the form of conditional prompt generation. DPL achieved a significant accuracy improvement with only training a lightweight prompt generator (a three-layer MLP), whose parameter is of equivalent scale to the classification projector in the previous DG literature. Combining dplshort~with CLIP provides surprising performance, raising the accuracy of zero-shot CLIP from 73.7% to 79.3% on several standard datasets, namely PACS, VLCS, OfficeHome, and TerraIncognita. We hope the simplicity and success of our approach lead to broader adoption and analysis of foundation models in the domain generalization field. Our code is available at https://github.com/shogi880/DPLCLIP.