# TMT: Cross-domain Semantic Segmentation with Region-adaptive Transferability Estimation

arXiv: https://arxiv.org/abs/2504.05774 · HTML: https://mlsanity.com/paper/2504.05774/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 17 of its 23 references that are in the map.
- First posted on arXiv 8 Apr 2025.

## Builds on (most cited first, 17 of 17)

- [Mask R-CNN](https://mlsanity.com/paper/1703.06870/) (2017)
- [The Cityscapes Dataset for Semantic Urban Scene Understanding](https://mlsanity.com/paper/1604.01685/) (2016)
- [Domain-Adversarial Training of Neural Networks](https://mlsanity.com/paper/1505.07818/) (2015)
- [Masked-attention Mask Transformer for Universal Image Segmentation](https://mlsanity.com/paper/2112.01527/) (2021)
- [BDD100K: A Diverse Driving Dataset for Heterogeneous Multitask Learning](https://mlsanity.com/paper/1805.04687/) (2018)
- [Unified Perceptual Parsing for Scene Understanding](https://mlsanity.com/paper/1807.10221/) (2018)
- [Playing for Data: Ground Truth from Computer Games](https://mlsanity.com/paper/1608.02192/) (2016)
- [MIC: Masked Image Consistency for Context-Enhanced Domain Adaptation](https://mlsanity.com/paper/2212.01322/) (2022)
- [SpotTune: Transfer Learning through Adaptive Fine-tuning](https://mlsanity.com/paper/1811.08737/) (2018)
- [HRDA: Context-Aware High-Resolution Domain-Adaptive Semantic Segmentation](https://mlsanity.com/paper/2204.13132/) (2022)
- [Lumina-mGPT: Illuminate Flexible Photorealistic Text-to-Image Generation with Multimodal Generative Pretraining](https://mlsanity.com/paper/2408.02657/) (2024)
- [STU-Net: Scalable and Transferable Medical Image Segmentation Models Empowered by Large-Scale Supervised Pre-training](https://mlsanity.com/paper/2304.06716/) (2023)
- [TVT: Transferable Vision Transformer for Unsupervised Domain Adaptation](https://mlsanity.com/paper/2108.05988/) (2021)
- [SparseViT: Revisiting Activation Sparsity for Efficient High-Resolution Vision Transformer](https://mlsanity.com/paper/2303.17605/) (2023)
- [Transferability-Guided Cross-Domain Cross-Task Transfer Learning](https://mlsanity.com/paper/2207.05510/) (2022)
- [Implicit and Explicit Language Guidance for Diffusion-based Visual Perception](https://mlsanity.com/paper/2404.07600/) (2024)
- [Not All Pixels Are Equal: Learning Pixel Hardness for Semantic Segmentation](https://mlsanity.com/paper/2305.08462/) (2023)

## Abstract (arXiv)

Recent advances in Vision Transformers (ViTs) have significantly advanced semantic segmentation performance. However, their adaptation to new target domains remains challenged by distribution shifts, which often disrupt global attention mechanisms. While existing global and patch-level adaptation methods offer some improvements, they overlook the spatially varying transferability inherent in different image regions. To address this, we propose the Transferable Mask Transformer (TMT), a region-adaptive framework designed to enhance cross-domain representation learning through transferability guidance. First, we dynamically partition the image into coherent regions, grouped by structural and semantic similarity, and estimates their domain transferability at a localized level. Then, we incorporate region-level transferability maps directly into the self-attention mechanism of ViTs, allowing the model to adaptively focus attention on areas with lower transferability and higher semantic uncertainty. Extensive experiments across 20 diverse cross-domain settings demonstrate that TMT not only mitigates the performance degradation typically associated with domain shift but also consistently outperforms existing approaches.