# When to Adapt: Multi-Signal Domain Shift Detection for Efficient Training-Free Adaptation in Open-Vocabulary Segmentation

arXiv: https://arxiv.org/abs/2609.37602 · HTML: https://mlsanity.com/paper/2609.37602/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 22 of its 27 references that are in the map.
- First posted on arXiv 29 Sep 2026.

## Builds on (most cited first, 22 of 22)

- [Learning Transferable Visual Models From Natural Language Supervision](https://mlsanity.com/paper/2103.00020/) (2021)
- [LoRA: Low-Rank Adaptation of Large Language Models](https://mlsanity.com/paper/2106.09685/) (2021)
- [ScanNet: Richly-annotated 3D Reconstructions of Indoor Scenes](https://mlsanity.com/paper/1702.04405/) (2017)
- [Robust fine-tuning of zero-shot models](https://mlsanity.com/paper/2109.01903/) (2021)
- [ScanNet++: A High-Fidelity Dataset of 3D Indoor Scenes](https://mlsanity.com/paper/2308.11417/) (2023)
- [Continual Test-Time Domain Adaptation](https://mlsanity.com/paper/2203.13591/) (2022)
- [EcoTTA: Memory-Efficient Continual Test-time Adaptation via Self-distilled Regularization](https://mlsanity.com/paper/2303.01904/) (2023)
- [SED: A Simple Encoder-Decoder for Open-Vocabulary Semantic Segmentation](https://mlsanity.com/paper/2311.15537/) (2023)
- [ViDA: Homeostatic Visual Domain Adapter for Continual Test Time Adaptation](https://mlsanity.com/paper/2306.04344/) (2023)
- [Continual Test-time Domain Adaptation via Dynamic Sample Selection](https://mlsanity.com/paper/2310.03335/) (2023)
- [MUSES: The Multi-Sensor Semantic Perception Dataset for Driving under Uncertainty](https://mlsanity.com/paper/2401.12761/) (2024)
- [Survey on Unsupervised Domain Adaptation for Semantic Segmentation for Visual Perception in Automated Driving](https://mlsanity.com/paper/2304.11928/) (2023)
- [Artificial Intelligence for Long-Term Robot Autonomy: A Survey](https://mlsanity.com/paper/1807.05196/) (2018)
- [Online Domain Adaptation for Semantic Segmentation in Ever-Changing Conditions](https://mlsanity.com/paper/2207.10667/) (2022)
- [Training-Free Model Merging for Multi-target Domain Adaptation](https://mlsanity.com/paper/2407.13771/) (2024)
- [To Adapt or Not to Adapt? Real-Time Adaptation for Semantic Segmentation](https://mlsanity.com/paper/2307.15063/) (2023)
- [Continual Adaptation of Semantic Segmentation using Complementary 2D-3D Data Representations](https://mlsanity.com/paper/2111.02156/) (2021)
- [Semantic Library Adaptation: LoRA Retrieval and Fusion for Open-Vocabulary Semantic Segmentation](https://mlsanity.com/paper/2503.21780/) (2025)
- [Controllable Continual Test-Time Adaptation](https://mlsanity.com/paper/2405.14602/) (2024)
- [Hybrid-TTA: Continual Test-time Adaptation via Dynamic Domain Shift Detection](https://mlsanity.com/paper/2409.08566/) (2024)
- [EmbodiTTA: Resource-Efficient Test-Time Adaptation for Embodied Visual Systems](https://mlsanity.com/paper/2505.00986/) (2025)
- [Instance-Guided Unsupervised Domain Adaptation for Robotic Semantic Segmentation](https://mlsanity.com/paper/2602.01389/) (2026)

## Abstract (arXiv)

Robust and reliable perception is essential for autonomous robots operating in real-world environments, particularly in long-term missions where environmental conditions may change significantly over time. Although recent advances in Visual Foundation Models (VFMs) have improved open-vocabulary semantic segmentation, these models can still suffer from domain shift, which can significantly degrade performance if they are not adapted to the current environment. Training-free domain adaptation is a relevant paradigm for adaptation, consisting of adjusting the model online using lightweight adapters. Recent approaches apply this on a per-frame basis, which is impractical for deployments on resource-constrained robotic hardware. To tackle this, we propose a multi-signal domain shift detection method for training-free continual test-time adaptation (TF-CTTA) in open-vocabulary segmentation. Our method leverages temporal coherence across consecutive frames by monitoring and combining complementary aspects of domain shift (visual change, adapter mismatch, and semantic drift) to trigger adaptation only when needed. We validate our approach on a benchmark including indoor and outdoor environments and using real robotic data. We demonstrate that our approach maintains segmentation accuracy while substantially reducing adaptations, making training-free adaptation practical and feasible for long-term, real-world robotic deployments.