# Vision-aware Multimodal Prompt Tuning for Uploadable Multi-source Few-shot Domain Adaptation

arXiv: https://arxiv.org/abs/2503.06106 · HTML: https://mlsanity.com/paper/2503.06106/

## Key facts

- Cited by 3 papers in the map (data through 14 Oct 2026).
- Builds on 21 of its 40 references that are in the map.
- First posted on arXiv 8 Mar 2025.

## Builds on (most cited first, 21 of 21)

- [Learning Transferable Visual Models From Natural Language Supervision](https://mlsanity.com/paper/2103.00020/) (2021)
- [An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale](https://mlsanity.com/paper/2010.11929/) (2020)
- [Communication-Efficient Learning of Deep Networks from Decentralized Data](https://mlsanity.com/paper/1602.05629/) (2016)
- [Prefix-Tuning: Optimizing Continuous Prompts for Generation](https://mlsanity.com/paper/2101.00190/) (2021)
- [The Power of Scale for Parameter-Efficient Prompt Tuning](https://mlsanity.com/paper/2104.08691/) (2021)
- [Moment Matching for Multi-Source Domain Adaptation](https://mlsanity.com/paper/1812.01754/) (2018)
- [P-Tuning v2: Prompt Tuning Can Be Comparable to Fine-tuning Universally Across Scales and Tasks](https://mlsanity.com/paper/2110.07602/) (2021)
- [Semi-supervised Domain Adaptation via Minimax Entropy](https://mlsanity.com/paper/1904.06487/) (2019)
- [Prompting Visual-Language Models for Efficient Video Understanding](https://mlsanity.com/paper/2112.04478/) (2021)
- [Generative Moment Matching Networks](https://mlsanity.com/paper/1502.02761/) (2015)
- [CPT: Colorful Prompt Tuning for Pre-trained Vision-Language Models](https://mlsanity.com/paper/2109.11797/) (2021)
- [Unified Vision and Language Prompt Learning](https://mlsanity.com/paper/2210.07225/) (2022)
- [Deep Cocktail Network: Multi-source Unsupervised Domain Adaptation with Category Shift](https://mlsanity.com/paper/1803.00830/) (2018)
- [AD-CLIP: Adapting Domains in Prompt Space Using CLIP](https://mlsanity.com/paper/2308.05659/) (2023)
- [Implicit Class-Conditioned Domain Alignment for Unsupervised Domain Adaptation](https://mlsanity.com/paper/2006.04996/) (2020)
- [Your Classifier can Secretly Suffice Multi-Source Domain Adaptation](https://mlsanity.com/paper/2103.11169/) (2021)
- [Aligning Domain-specific Distribution and Classifier for Cross-domain Classification from Multiple Sources](https://mlsanity.com/paper/2201.01003/) (2022)
- [Cross-domain Self-supervised Learning for Domain Adaptation with Few Source Labels](https://mlsanity.com/paper/2003.08264/) (2020)
- [Retrieval-Augmented Mixture of LoRA Experts for Uploadable Machine Learning](https://mlsanity.com/paper/2406.16989/) (2024)
- [LanDA: Language-Guided Multi-Source Domain Adaptation](https://mlsanity.com/paper/2401.14148/) (2024)
- [Multi-source Few-shot Domain Adaptation](https://mlsanity.com/paper/2109.12391/) (2021)

## Built on by (most cited first, 3 of 3)

- [CLIP-Powered Domain Generalization and Domain Adaptation: A Comprehensive Survey](https://mlsanity.com/paper/2504.14280/) (2025)
- [Visual Saliency Steering Distillation for Multimodal Chain-of-Thought Reasoning](https://mlsanity.com/paper/2607.22013/) (2026)
- [DICA: Dual-Indicator Guided Contrastive Alignment in Multimodal Large Language Models](https://mlsanity.com/paper/2607.23944/) (2026)

## Abstract (arXiv)

Conventional multi-source domain few-shot adaptation (MFDA) faces the challenge of further reducing the load on edge-side devices in low-resource scenarios. Considering the native language-supervised advantage of CLIP and the plug-and-play nature of prompt to transfer CLIP efficiently, this paper introduces an uploadable multi-source few-shot domain adaptation (UMFDA) schema. It belongs to a decentralized edge collaborative learning in the edge-side models that must maintain a low computational load. And only a limited amount of annotations in source domain data is provided, with most of the data being unannotated. Further, this paper proposes a vision-aware multimodal prompt tuning framework (VAMP) under the decentralized schema, where the vision-aware prompt guides the text domain-specific prompt to maintain semantic discriminability and perceive the domain information. The cross-modal semantic and domain distribution alignment losses optimize each edge-side model, while text classifier consistency and semantic diversity losses promote collaborative learning among edge-side models. Extensive experiments were conducted on OfficeHome and DomainNet datasets to demonstrate the effectiveness of the proposed VAMP in the UMFDA, which outperformed the previous prompt tuning methods.