# OT-VP: Optimal Transport-guided Visual Prompting for Test-Time Adaptation

arXiv: https://arxiv.org/abs/2407.09498 · HTML: https://mlsanity.com/paper/2407.09498/

## Key facts

- Cited by 17 papers in the map (data through 14 Oct 2026).
- Builds on 38 of its 57 references that are in the map.
- First posted on arXiv 12 Jun 2024.

## Builds on (most cited first, 30 of 38)

- [Learning Transferable Visual Models From Natural Language Supervision](https://mlsanity.com/paper/2103.00020/) (2021)
- [An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale](https://mlsanity.com/paper/2010.11929/) (2020)
- [PyTorch: An Imperative Style, High-Performance Deep Learning Library](https://mlsanity.com/paper/1912.01703/) (2019)
- [Swin Transformer: Hierarchical Vision Transformer using Shifted Windows](https://mlsanity.com/paper/2103.14030/) (2021)
- [Overcoming catastrophic forgetting in neural networks](https://mlsanity.com/paper/1612.00796/) (2016)
- [Benchmarking Neural Network Robustness to Common Corruptions and Perturbations](https://mlsanity.com/paper/1903.12261/) (2019)
- [Sinkhorn Distances: Lightspeed Computation of Optimal Transportation Distances](https://mlsanity.com/paper/1306.0895/) (2013)
- [Visual Prompt Tuning](https://mlsanity.com/paper/2203.12119/) (2022)
- [Do ImageNet Classifiers Generalize to ImageNet?](https://mlsanity.com/paper/1902.10811/) (2019)
- [WILDS: A Benchmark of in-the-Wild Distribution Shifts](https://mlsanity.com/paper/2012.07421/) (2020)
- [Deep Hashing Network for Unsupervised Domain Adaptation](https://mlsanity.com/paper/1706.07522/) (2017)
- [In Search of Lost Domain Generalization](https://mlsanity.com/paper/2007.01434/) (2020)
- [Deeper, Broader and Artier Domain Generalization](https://mlsanity.com/paper/1710.03077/) (2017)
- [Learning to Prompt for Continual Learning](https://mlsanity.com/paper/2112.08654/) (2021)
- [Do We Really Need to Access the Source Data? Source Hypothesis Transfer for Unsupervised Domain Adaptation](https://mlsanity.com/paper/2002.08546/) (2020)
- [Computational Optimal Transport](https://mlsanity.com/paper/1803.00567/) (2018)
- [Domain Generalization: A Survey](https://mlsanity.com/paper/2103.02503/) (2021)
- [Do Vision Transformers See Like Convolutional Neural Networks?](https://mlsanity.com/paper/2108.08810/) (2021)
- [Optimal Transport for Domain Adaptation](https://mlsanity.com/paper/1507.00504/) (2015)
- [Intriguing Properties of Vision Transformers](https://mlsanity.com/paper/2105.10497/) (2021)
- [Improving robustness against common corruptions by covariate shift adaptation](https://mlsanity.com/paper/2006.16971/) (2020)
- [Towards Stable Test-Time Adaptation in Dynamic Wild World](https://mlsanity.com/paper/2302.12400/) (2023)
- [A Comprehensive Survey on Test-Time Adaptation under Distribution Shifts](https://mlsanity.com/paper/2303.15361/) (2023)
- [Certifying Some Distributional Robustness with Principled Adversarial Training](https://mlsanity.com/paper/1710.10571/) (2017)
- [Source Data-absent Unsupervised Domain Adaptation through Hypothesis Transfer and Labeling Transfer](https://mlsanity.com/paper/2012.07297/) (2020)
- [Geometric Dataset Distances via Optimal Transport](https://mlsanity.com/paper/2002.02923/) (2020)
- [The Norm Must Go On: Dynamic Unsupervised Domain Adaptation by Normalization](https://mlsanity.com/paper/2112.00463/) (2021)
- [Domain Adaptation via Prompt Learning](https://mlsanity.com/paper/2202.06687/) (2022)
- [Align Your Prompts: Test-Time Prompting with Distribution Alignment for Zero-Shot Generalization](https://mlsanity.com/paper/2311.01459/) (2023)
- [Decorate the Newcomers: Visual Domain Prompt for Continual Test Time Adaptation](https://mlsanity.com/paper/2212.04145/) (2022)

## Built on by (most cited first, 17 of 17)

- [DPCore: Dynamic Prompt Coreset for Continual Test-Time Adaptation](https://mlsanity.com/paper/2406.10737/) (2024)
- [Prompt-based Adaptation in Large-scale Vision Models: A Survey](https://mlsanity.com/paper/2510.13219/) (2025)
- [Visual Variational Autoencoder Prompt Tuning](https://mlsanity.com/paper/2503.17650/) (2025)
- [Doctor Approved: Generating Medically Accurate Skin Disease Images through AI-Expert Feedback](https://mlsanity.com/paper/2506.12323/) (2025)
- [All You Need is One: Capsule Prompt Tuning with a Single Vector](https://mlsanity.com/paper/2510.16670/) (2025)
- [Revisiting semi-supervised learning in the era of foundation models](https://mlsanity.com/paper/2503.09707/) (2025)
- [Semantic Anchor Transport: Robust Test-Time Adaptation for Vision-Language Models](https://mlsanity.com/paper/2411.17002/) (2024)
- [ZeroSiam: An Efficient Asymmetry for Test-Time Entropy Optimization without Collapse](https://mlsanity.com/paper/2509.23183/) (2025)
- [Class-aware Domain Knowledge Fusion and Fission for Continual Test-Time Adaptation](https://mlsanity.com/paper/2510.12150/) (2025)
- [Lessons and Open Questions from a Unified Study of Camera-Trap Species Recognition Over Time](https://mlsanity.com/paper/2603.20509/) (2026)
- [PAID: Pairwise Angular-Invariant Decomposition for Continual Test-Time Adaptation](https://mlsanity.com/paper/2506.02453/) (2025)
- [Adapting in the Dark: Efficient and Stable Test-Time Adaptation for Black-Box Models](https://mlsanity.com/paper/2604.15609/) (2026)
- [Continual Test-Time Adaptation in Computer Vision: Methods, Benchmarks, and Future Directions](https://mlsanity.com/paper/2607.08164/) (2026)
- [Hierarchical Adaptive networks with Task vectors for Test-Time Adaptation](https://mlsanity.com/paper/2508.09223/) (2025)
- [Beyond Low-Rank: Low-Rank Sparse Prompting via Spiking Neural Network and Prompt Factorization](https://mlsanity.com/paper/2606.01945/) (2026)
- [Less Tokens, Better Forecasts: Sparse Residual Routing for Efficient Weather Prediction](https://mlsanity.com/paper/2607.02829/) (2026)
- [How Medical VLMs Underutilize Their Vision Encoders: A Dermatology Perspective](https://mlsanity.com/paper/2609.36557/) (2026)

## Abstract (arXiv)

Vision Transformers (ViTs) have demonstrated remarkable capabilities in learning representations, but their performance is compromised when applied to unseen domains. Previous methods either engage in prompt learning during the training phase or modify model parameters at test time through entropy minimization. The former often overlooks unlabeled target data, while the latter doesn't fully address domain shifts. In this work, our approach, Optimal Transport-guided Test-Time Visual Prompting (OT-VP), handles these problems by leveraging prompt learning at test time to align the target and source domains without accessing the training process or altering pre-trained model parameters. This method involves learning a universal visual prompt for the target domain by optimizing the Optimal Transport distance.OT-VP, with only four learned prompt tokens, exceeds state-of-the-art performance across three stylistic datasets-PACS, VLCS, OfficeHome, and one corrupted dataset ImageNet-C. Additionally, OT-VP operates efficiently, both in terms of memory and computation, and is adaptable for extension to online settings.