# Prompt Vision Transformer for Domain Generalization

arXiv: https://arxiv.org/abs/2208.08914 · HTML: https://mlsanity.com/paper/2208.08914/

## Key facts

- Cited by 52 papers in the map (data through 14 Oct 2026).
- Builds on 37 of its 47 references that are in the map.
- First posted on arXiv 18 Aug 2022.

## Builds on (most cited first, 30 of 37)

- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale](https://mlsanity.com/paper/2010.11929/) (2020)
- [PyTorch: An Imperative Style, High-Performance Deep Learning Library](https://mlsanity.com/paper/1912.01703/) (2019)
- [Swin Transformer: Hierarchical Vision Transformer using Shifted Windows](https://mlsanity.com/paper/2103.14030/) (2021)
- [Model-Agnostic Meta-Learning for Fast Adaptation of Deep Networks](https://mlsanity.com/paper/1703.03400/) (2017)
- [Fully Convolutional Networks for Semantic Segmentation](https://mlsanity.com/paper/1411.4038/) (2014)
- [You Only Look Once: Unified, Real-Time Object Detection](https://mlsanity.com/paper/1506.02640/) (2015)
- [Domain-Adversarial Training of Neural Networks](https://mlsanity.com/paper/1505.07818/) (2015)
- [Learning to Prompt for Vision-Language Models](https://mlsanity.com/paper/2109.01134/) (2021)
- [Matching Networks for One Shot Learning](https://mlsanity.com/paper/1606.04080/) (2016)
- [Unsupervised Representation Learning with Deep Convolutional Generative Adversarial Networks](https://mlsanity.com/paper/1511.06434/) (2015)
- [Arbitrary Style Transfer in Real-time with Adaptive Instance Normalization](https://mlsanity.com/paper/1703.06868/) (2017)
- [Invariant Risk Minimization](https://mlsanity.com/paper/1907.02893/) (2019)
- [Moment Matching for Multi-Source Domain Adaptation](https://mlsanity.com/paper/1812.01754/) (2018)
- [Distributionally Robust Neural Networks for Group Shifts: On the Importance of Regularization for Worst-Case Generalization](https://mlsanity.com/paper/1911.08731/) (2019)
- [Deep CORAL: Correlation Alignment for Deep Domain Adaptation](https://mlsanity.com/paper/1607.01719/) (2016)
- [Deep Hashing Network for Unsupervised Domain Adaptation](https://mlsanity.com/paper/1706.07522/) (2017)
- [In Search of Lost Domain Generalization](https://mlsanity.com/paper/2007.01434/) (2020)
- [Deeper, Broader and Artier Domain Generalization](https://mlsanity.com/paper/1710.03077/) (2017)
- [Deep Networks with Stochastic Depth](https://mlsanity.com/paper/1603.09382/) (2016)
- [Learning to Generalize: Meta-Learning for Domain Generalization](https://mlsanity.com/paper/1710.03463/) (2017)
- [P-Tuning v2: Prompt Tuning Can Be Comparable to Fine-tuning Universally Across Scales and Tasks](https://mlsanity.com/paper/2110.07602/) (2021)
- [Do Vision Transformers See Like Convolutional Neural Networks?](https://mlsanity.com/paper/2108.08810/) (2021)
- [Domain Generalization by Solving Jigsaw Puzzles](https://mlsanity.com/paper/1903.06864/) (2019)
- [Intriguing Properties of Vision Transformers](https://mlsanity.com/paper/2105.10497/) (2021)
- [SWAD: Domain Generalization by Seeking Flat Minima](https://mlsanity.com/paper/2102.08604/) (2021)
- [Self-Challenging Improves Cross-Domain Generalization](https://mlsanity.com/paper/2007.02454/) (2020)
- [Unified Deep Supervised Domain Adaptation and Generalization](https://mlsanity.com/paper/1709.10190/) (2017)
- [Domain Separation Networks](https://mlsanity.com/paper/1608.06019/) (2016)

## Built on by (most cited first, 30 of 52)

- [A Comprehensive Survey on Test-Time Adaptation under Distribution Shifts](https://mlsanity.com/paper/2303.15361/) (2023)
- [Visual Prompt Multi-Modal Tracking](https://mlsanity.com/paper/2303.10826/) (2023)
- [Prompting Segmentation with Sound Is Generalizable Audio-Visual Source Localizer](https://mlsanity.com/paper/2309.07929/) (2023)
- [Learning Domain Invariant Prompt for Vision-Language Models](https://mlsanity.com/paper/2212.04196/) (2022)
- [Temporal Adaptive RGBT Tracking with Modality Prompt](https://mlsanity.com/paper/2401.01244/) (2024)
- [Visual Instance-aware Prompt Tuning](https://mlsanity.com/paper/2507.07796/) (2025)
- [ALOFT: A Lightweight MLP-like Architecture with Dynamic Low-frequency Transform for Domain Generalization](https://mlsanity.com/paper/2303.11674/) (2023)
- [What can a cook in Italy teach a mechanic in India? Action Recognition Generalisation Over Scenarios and Locations](https://mlsanity.com/paper/2306.08713/) (2023)
- [SelfPromer: Self-Prompt Dehazing Transformers with Depth-Consistency](https://mlsanity.com/paper/2303.07033/) (2023)
- [Soft Prompt Generation for Domain Generalization](https://mlsanity.com/paper/2404.19286/) (2024)
- [Visual Prompting Upgrades Neural Network Sparsification: A Data-Model Perspective](https://mlsanity.com/paper/2312.01397/) (2023)
- [Adapting to Distribution Shift by Visual Domain Prompt Generation](https://mlsanity.com/paper/2405.02797/) (2024)
- [DGMamba: Domain Generalization via Generalized State Space Model](https://mlsanity.com/paper/2404.07794/) (2024)
- [SCT: A Simple Baseline for Parameter-Efficient Fine-Tuning via Salient Channels](https://mlsanity.com/paper/2309.08513/) (2023)
- [DAM: Dynamic Adapter Merging for Continual Video QA Learning](https://mlsanity.com/paper/2403.08755/) (2024)
- [ProSFDA: Prompt Learning based Source-free Domain Adaptation for Medical Image Segmentation](https://mlsanity.com/paper/2211.11514/) (2022)
- [OT-VP: Optimal Transport-guided Visual Prompting for Test-Time Adaptation](https://mlsanity.com/paper/2407.09498/) (2024)
- [DCPT: Darkness Clue-Prompted Tracking in Nighttime UAVs](https://mlsanity.com/paper/2309.10491/) (2023)
- [A Systematic Survey of Prompt Engineering on Vision-Language Foundation Models](https://mlsanity.com/paper/2307.12980/) (2023)
- [Tri-Level Navigator: LLM-Empowered Tri-Level Learning for Time Series OOD Generalization](https://mlsanity.com/paper/2410.07018/) (2024)
- [Defending Multimodal Backdoored Models by Repulsive Visual Prompt Tuning](https://mlsanity.com/paper/2412.20392/) (2024)
- [POV: Prompt-Oriented View-Agnostic Learning for Egocentric Hand-Object Interaction in the Multi-View World](https://mlsanity.com/paper/2403.05856/) (2024)
- [Seeking Consistent Flat Minima for Better Domain Generalization via Refining Loss Landscapes](https://mlsanity.com/paper/2412.13573/) (2024)
- [FlexAvatar: Learning Complete 3D Head Avatars with Partial Supervision](https://mlsanity.com/paper/2512.15599/) (2025)
- [EPVT: Environment-aware Prompt Vision Transformer for Domain Generalization in Skin Lesion Recognition](https://mlsanity.com/paper/2304.01508/) (2023)
- [Prompt-driven Latent Domain Generalization for Medical Image Classification](https://mlsanity.com/paper/2401.03002/) (2024)
- [HCVP: Leveraging Hierarchical Contrastive Visual Prompt for Domain Generalization](https://mlsanity.com/paper/2401.09716/) (2024)
- [Egocentric zone-aware action recognition across environments](https://mlsanity.com/paper/2409.14205/) (2024)
- [Revisit Parameter-Efficient Transfer Learning: A Two-Stage Paradigm](https://mlsanity.com/paper/2303.07910/) (2023)
- [FocusFlow: Boosting Key-Points Optical Flow Estimation for Autonomous Driving](https://mlsanity.com/paper/2308.07104/) (2023)

## Abstract (arXiv)

Though vision transformers (ViTs) have exhibited impressive ability for representation learning, we empirically find that they cannot generalize well to unseen domains with previous domain generalization algorithms. In this paper, we propose a novel approach DoPrompt based on prompt learning to embed the knowledge of source domains in domain prompts for target domain prediction. Specifically, domain prompts are prepended before ViT input tokens from the corresponding source domain. Each domain prompt learns domain-specific knowledge efficiently since it is optimized only for one domain. Meanwhile, we train a prompt adapter to produce a suitable prompt for each input image based on the learned source domain prompts. At test time, the adapted prompt generated by the prompt adapter can exploit the similarity between the feature of the out-of-domain image and source domains to properly integrate the source domain knowledge. Extensive experiments are conducted on four benchmark datasets. Our approach achieves 1.4% improvements in the averaged accuracy, which is 3.5 times the improvement of the state-of-the-art algorithm with a ViT backbone.