# Decorate the Newcomers: Visual Domain Prompt for Continual Test Time Adaptation

arXiv: https://arxiv.org/abs/2212.04145 · HTML: https://mlsanity.com/paper/2212.04145/

## Key facts

- Cited by 98 papers in the map (data through 14 Oct 2026).
- Builds on 24 of its 33 references that are in the map.
- First posted on arXiv 8 Dec 2022.

## Builds on (most cited first, 24 of 24)

- [Overcoming catastrophic forgetting in neural networks](https://mlsanity.com/paper/1612.00796/) (2016)
- [Wide Residual Networks](https://mlsanity.com/paper/1605.07146/) (2016)
- [Learning to Prompt for Vision-Language Models](https://mlsanity.com/paper/2109.01134/) (2021)
- [Pre-train, Prompt, and Predict: A Systematic Survey of Prompting Methods in Natural Language Processing](https://mlsanity.com/paper/2107.13586/) (2021)
- [Visual Prompt Tuning](https://mlsanity.com/paper/2203.12119/) (2022)
- [Conditional Prompt Learning for Vision-Language Models](https://mlsanity.com/paper/2203.05557/) (2022)
- [Experience Replay for Continual Learning](https://mlsanity.com/paper/1811.11682/) (2018)
- [Learning to Prompt for Continual Learning](https://mlsanity.com/paper/2112.08654/) (2021)
- [AugMix: A Simple Data Processing Method to Improve Robustness and Uncertainty](https://mlsanity.com/paper/1912.02781/) (2019)
- [Do We Really Need to Access the Source Data? Source Hypothesis Transfer for Unsupervised Domain Adaptation](https://mlsanity.com/paper/2002.08546/) (2020)
- [RobustBench: a standardized adversarial robustness benchmark](https://mlsanity.com/paper/2010.09670/) (2020)
- [Continual Test-Time Domain Adaptation](https://mlsanity.com/paper/2203.13591/) (2022)
- [Improving robustness against common corruptions by covariate shift adaptation](https://mlsanity.com/paper/2006.16971/) (2020)
- [Exploring Visual Prompts for Adapting Large-Scale Models](https://mlsanity.com/paper/2203.17274/) (2022)
- [Contrastive Test-Time Adaptation](https://mlsanity.com/paper/2204.10377/) (2022)
- [Real-World Robot Learning with Masked Visual Pre-training](https://mlsanity.com/paper/2210.03109/) (2022)
- [Evaluating Prediction-Time Batch Normalization for Robustness under Covariate Shift](https://mlsanity.com/paper/2006.10963/) (2020)
- [Parameter-free Online Test-time Adaptation](https://mlsanity.com/paper/2201.05718/) (2022)
- [Generalized Source-free Domain Adaptation](https://mlsanity.com/paper/2108.01614/) (2021)
- [Align and Prompt: Video-and-Language Pre-training with Entity Prompts](https://mlsanity.com/paper/2112.09583/) (2021)
- [Domain Adaptation via Prompt Learning](https://mlsanity.com/paper/2202.06687/) (2022)
- [Feature Importance-aware Transferable Adversarial Attacks](https://mlsanity.com/paper/2107.14185/) (2021)
- [Adversarial Patch Attacks and Defences in Vision-Based Tasks: A Survey](https://mlsanity.com/paper/2206.08304/) (2022)
- [Limitations of Post-Hoc Feature Alignment for Robustness](https://mlsanity.com/paper/2103.05898/) (2021)

## Built on by (most cited first, 30 of 98)

- [A Comprehensive Survey on Test-Time Adaptation under Distribution Shifts](https://mlsanity.com/paper/2303.15361/) (2023)
- [Personalize Segment Anything Model with One Shot](https://mlsanity.com/paper/2305.03048/) (2023)
- [Continual Learning with Pre-Trained Models: A Survey](https://mlsanity.com/paper/2401.16386/) (2024)
- [Not All Features Matter: Enhancing Few-shot CLIP with Adaptive Prior Refinement](https://mlsanity.com/paper/2304.01195/) (2023)
- [A Comprehensive Survey of Forgetting in Deep Learning Beyond Continual Learning](https://mlsanity.com/paper/2307.09218/) (2023)
- [ViDA: Homeostatic Visual Domain Adapter for Continual Test Time Adaptation](https://mlsanity.com/paper/2306.04344/) (2023)
- [Improved Zero-Shot Classification by Adapting VLMs with Text Descriptions](https://mlsanity.com/paper/2401.02460/) (2024)
- [In Search of Lost Online Test-time Adaptation: A Survey](https://mlsanity.com/paper/2310.20199/) (2023)
- [Towards Open-Set Test-Time Adaptation Utilizing the Wisdom of Crowds in Entropy Minimization](https://mlsanity.com/paper/2308.06879/) (2023)
- [Continual Test-time Domain Adaptation via Dynamic Sample Selection](https://mlsanity.com/paper/2310.03335/) (2023)
- [Visual Tuning](https://mlsanity.com/paper/2305.06061/) (2023)
- [DPCore: Dynamic Prompt Coreset for Continual Test-Time Adaptation](https://mlsanity.com/paper/2406.10737/) (2024)
- [Domain-Agnostic Mutual Prompting for Unsupervised Domain Adaptation](https://mlsanity.com/paper/2403.02899/) (2024)
- [Continual-MAE: Adaptive Distribution Masked Autoencoders for Continual Test-Time Adaptation](https://mlsanity.com/paper/2312.12480/) (2023)
- [Each Test Image Deserves A Specific Prompt: Continual Test-Time Adaptation for 2D Medical Image Segmentation](https://mlsanity.com/paper/2311.18363/) (2023)
- [Exploring Sparse Visual Prompt for Domain Adaptive Dense Prediction](https://mlsanity.com/paper/2303.09792/) (2023)
- [FreeKD: Knowledge Distillation via Semantic Frequency Prompt](https://mlsanity.com/paper/2311.12079/) (2023)
- [BECoTTA: Input-dependent Online Blending of Experts for Continual Test-time Adaptation](https://mlsanity.com/paper/2402.08712/) (2024)
- [SelfPromer: Self-Prompt Dehazing Transformers with Depth-Consistency](https://mlsanity.com/paper/2303.07033/) (2023)
- [Adapting to Distribution Shift by Visual Domain Prompt Generation](https://mlsanity.com/paper/2405.02797/) (2024)
- [Advances in Multimodal Adaptation and Generalization: From Traditional Approaches to Foundation Models](https://mlsanity.com/paper/2501.18592/) (2025)
- [Multi-Modal Continual Test-Time Adaptation for 3D Semantic Segmentation](https://mlsanity.com/paper/2303.10457/) (2023)
- [Prompted Contrast with Masked Motion Modeling: Towards Versatile 3D Action Representation Learning](https://mlsanity.com/paper/2308.03975/) (2023)
- [Seeing the Unseen: A Frequency Prompt Guided Transformer for Image Restoration](https://mlsanity.com/paper/2404.00288/) (2024)
- [Benchmarking Test-Time Adaptation against Distribution Shifts in Image Classification](https://mlsanity.com/paper/2307.03133/) (2023)
- [Test-Time Degradation Adaptation for Open-Set Image Restoration](https://mlsanity.com/paper/2312.02197/) (2023)
- [OT-VP: Optimal Transport-guided Visual Prompting for Test-Time Adaptation](https://mlsanity.com/paper/2407.09498/) (2024)
- [Ranked Entropy Minimization for Continual Test-Time Adaptation](https://mlsanity.com/paper/2505.16441/) (2025)
- [One is Plenty: A Polymorphic Feature Interpreter for Immutable Heterogeneous Collaborative Perception](https://mlsanity.com/paper/2411.16799/) (2024)
- [PromptRR: Diffusion Models as Prompt Generators for Single Image Reflection Removal](https://mlsanity.com/paper/2402.02374/) (2024)

## Abstract (arXiv)

Continual Test-Time Adaptation (CTTA) aims to adapt the source model to continually changing unlabeled target domains without access to the source data. Existing methods mainly focus on model-based adaptation in a self-training manner, such as predicting pseudo labels for new domain datasets. Since pseudo labels are noisy and unreliable, these methods suffer from catastrophic forgetting and error accumulation when dealing with dynamic data distributions. Motivated by the prompt learning in NLP, in this paper, we propose to learn an image-level visual domain prompt for target domains while having the source model parameters frozen. During testing, the changing target datasets can be adapted to the source model by reformulating the input data with the learned visual prompts. Specifically, we devise two types of prompts, i.e., domains-specific prompts and domains-agnostic prompts, to extract current domain knowledge and maintain the domain-shared knowledge in the continual adaptation. Furthermore, we design a homeostasis-based prompt adaptation strategy to suppress domain-sensitive parameters in domain-invariant prompts to learn domain-shared knowledge more effectively. This transition from the model-dependent paradigm to the model-free one enables us to bypass the catastrophic forgetting and error accumulation problems. Experiments show that our proposed method achieves significant performance gains over state-of-the-art methods on four widely-used benchmarks, including CIFAR-10C, CIFAR-100C, ImageNet-C, and VLCS datasets.