# TestMate: Test-Time Domain Adaptation Aided by Lightweight Vision Foundation Model

arXiv: https://arxiv.org/abs/2607.03810 · HTML: https://mlsanity.com/paper/2607.03810/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 28 of its 41 references that are in the map.
- First posted on arXiv 4 Jul 2026.

## Builds on (most cited first, 28 of 28)

- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [Segment Anything](https://mlsanity.com/paper/2304.02643/) (2023)
- [The Cityscapes Dataset for Semantic Urban Scene Understanding](https://mlsanity.com/paper/1604.01685/) (2016)
- [DeepLab: Semantic Image Segmentation with Deep Convolutional Nets, Atrous Convolution, and Fully Connected CRFs](https://mlsanity.com/paper/1606.00915/) (2016)
- [Rethinking Atrous Convolution for Semantic Image Segmentation](https://mlsanity.com/paper/1706.05587/) (2017)
- [Playing for Data: Ground Truth from Computer Games](https://mlsanity.com/paper/1608.02192/) (2016)
- [Simple Copy-Paste is a Strong Data Augmentation Method for Instance Segmentation](https://mlsanity.com/paper/2012.07177/) (2020)
- [Continual Test-Time Domain Adaptation](https://mlsanity.com/paper/2203.13591/) (2022)
- [Efficient Test-Time Model Adaptation without Forgetting](https://mlsanity.com/paper/2204.02610/) (2022)
- [Fast Segment Anything](https://mlsanity.com/paper/2306.12156/) (2023)
- [Source-Free Domain Adaptation for Semantic Segmentation](https://mlsanity.com/paper/2103.16372/) (2021)
- [A review of domain adaptation without target labels](https://mlsanity.com/paper/1901.05335/) (2019)
- [Model Adaptation: Historical Contrastive Learning for Unsupervised Domain Adaptation without Source Data](https://mlsanity.com/paper/2110.03374/) (2021)
- [Domain Adaptation for Semantic Segmentation with Maximum Squares Loss](https://mlsanity.com/paper/1909.13589/) (2019)
- [Multi-interactive Feature Learning and a Full-time Multi-modality Benchmark for Image Fusion and Segmentation](https://mlsanity.com/paper/2308.02097/) (2023)
- [EcoTTA: Memory-Efficient Continual Test-time Adaptation via Self-distilled Regularization](https://mlsanity.com/paper/2303.01904/) (2023)
- [RGB-T Object Tracking:Benchmark and Baseline](https://mlsanity.com/paper/1805.08982/) (2018)
- [Robust Mean Teacher for Continual and Gradual Test-Time Adaptation](https://mlsanity.com/paper/2211.13081/) (2022)
- [A Comprehensive Survey on Source-free Domain Adaptation](https://mlsanity.com/paper/2302.11803/) (2023)
- [Back to the Source: Diffusion-Driven Test-Time Adaptation](https://mlsanity.com/paper/2207.03442/) (2022)
- [VDM-DA: Virtual Domain Modeling for Source Data-free Domain Adaptation](https://mlsanity.com/paper/2103.14357/) (2021)
- [Source-Free Domain Adaptation with Frozen Multimodal Foundation Model](https://mlsanity.com/paper/2311.16510/) (2023)
- [Feature Alignment and Uniformity for Test Time Adaptation](https://mlsanity.com/paper/2303.10902/) (2023)
- [Deflating Dataset Bias Using Synthetic Data Augmentation](https://mlsanity.com/paper/2004.13866/) (2020)
- [Ev-TTA: Test-Time Adaptation for Event-Based Object Recognition](https://mlsanity.com/paper/2203.12247/) (2022)
- [GDA: Generalized Diffusion for Robust Test-time Adaptation](https://mlsanity.com/paper/2404.00095/) (2024)
- [SAM4UDASS: When SAM Meets Unsupervised Domain Adaptive Semantic Segmentation in Intelligent Vehicles](https://mlsanity.com/paper/2401.08604/) (2023)
- [Generalization in Neural Networks: A Broad Survey](https://mlsanity.com/paper/2209.01610/) (2022)

## Abstract (arXiv)

Test-Time Domain Adaptation (TTDA) aims to adapt Deep Neural Networks to distribution shifts using only streaming, unlabeled test data in real time. Current methods for semantic segmentation tasks suffer from critical limitations. Entropy minimization techniques require costly backpropagation, risking catastrophic forgetting and producing noisy segmentation boundaries. Memory-bank methods, while backpropagation-free, exhibit slow adaptation, requiring numerous samples to converge and struggle to handle continuous domain shifts. We introduce TestMate, a novel, real-time, and backpropagation-free TTDA framework that overcomes these issues. TestMate leverages generalization capability of a lightweight Visual Foundation Model to guide the adaptation. We use a zero-shot instance segmentation YOLOv8-seg based model to generate unlabeled mask proposals for objects and their parts at multiple scales in real time. These proposals are fused with the primary model via a heuristic, size-ordered competitive scheme, where small, high-confidence regions dominate and refine predictions in surrounding larger, less certain areas. This paremeter-free mechanism enables immediate adaptation from the first frame, inherently avoids catastrophic forgetting and effectively preserves fine object details and boundaries, even for small objects. TestMate can be used as a standalone, efficient refinement module or seamlessly integrated into existing TTDA methods to significantly boost their performance. We demonstrate state-of-the-art results across two benchmark datasets, proving TestMate's effectiveness in three distinct adaptation tasks: TTDA, Source-Free Domain Adaptation (SFDA), and online-TTDA. Code is available.