# Efficient Domain-Adaptive Multi-Task Dense Prediction with Vision Foundation Models

arXiv: https://arxiv.org/abs/2509.23626 · HTML: https://mlsanity.com/paper/2509.23626/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 39 of its 47 references that are in the map.
- First posted on arXiv 28 Sep 2025.

## Builds on (most cited first, 30 of 39)

- [Swin Transformer: Hierarchical Vision Transformer using Shifted Windows](https://mlsanity.com/paper/2103.14030/) (2021)
- [Segment Anything](https://mlsanity.com/paper/2304.02643/) (2023)
- [The Cityscapes Dataset for Semantic Urban Scene Understanding](https://mlsanity.com/paper/1604.01685/) (2016)
- [SAM 2: Segment Anything in Images and Videos](https://mlsanity.com/paper/2408.00714/) (2024)
- [Towards Robust Monocular Depth Estimation: Mixing Datasets for Zero-shot Cross-dataset Transfer](https://mlsanity.com/paper/1907.01341/) (2019)
- [Depth Anything V2](https://mlsanity.com/paper/2406.09414/) (2024)
- [Depth Anything: Unleashing the Power of Large-Scale Unlabeled Data](https://mlsanity.com/paper/2401.10891/) (2024)
- [Multi-Task Learning Using Uncertainty to Weigh Losses for Scene Geometry and Semantics](https://mlsanity.com/paper/1705.07115/) (2017)
- [Gradient Surgery for Multi-Task Learning](https://mlsanity.com/paper/2001.06782/) (2020)
- [GradNorm: Gradient Normalization for Adaptive Loss Balancing in Deep Multitask Networks](https://mlsanity.com/paper/1711.02257/) (2017)
- [End-to-End Multi-Task Learning with Attention](https://mlsanity.com/paper/1803.10704/) (2018)
- [Metric3D: Towards Zero-shot Metric 3D Prediction from A Single Image](https://mlsanity.com/paper/2307.10984/) (2023)
- [Virtual KITTI 2](https://mlsanity.com/paper/2001.10773/) (2020)
- [DAFormer: Improving Network Architectures and Training Strategies for Domain-Adaptive Semantic Segmentation](https://mlsanity.com/paper/2111.14887/) (2021)
- [Fast Segment Anything](https://mlsanity.com/paper/2306.12156/) (2023)
- [PAD-Net: Multi-Tasks Guided Prediction-and-Distillation Network for Simultaneous Depth Estimation and Scene Parsing](https://mlsanity.com/paper/1805.04409/) (2018)
- [MIC: Masked Image Consistency for Context-Enhanced Domain Adaptation](https://mlsanity.com/paper/2212.01322/) (2022)
- [FoundationStereo: Zero-Shot Stereo Matching](https://mlsanity.com/paper/2501.09898/) (2025)
- [HRDA: Context-Aware High-Resolution Domain-Adaptive Semantic Segmentation](https://mlsanity.com/paper/2204.13132/) (2022)
- [AdaShare: Learning What To Share For Efficient Deep Multi-Task Learning](https://mlsanity.com/paper/1911.12423/) (2019)
- [Pattern-Affinitive Propagation across Depth, Surface Normal and Semantic Segmentation](https://mlsanity.com/paper/1906.03525/) (2019)
- [Robust Learning Through Cross-Task Consistency](https://mlsanity.com/paper/2006.04096/) (2020)
- [Multi-Task Self-Training for Learning General Representations](https://mlsanity.com/paper/2108.11353/) (2021)
- [Domain Adaptive Semantic Segmentation with Self-Supervised Depth Estimation](https://mlsanity.com/paper/2104.13613/) (2021)
- [Marigold: Affordable Adaptation of Diffusion-Based Image Generators for Image Analysis](https://mlsanity.com/paper/2505.09358/) (2025)
- [Multi-Task Dense Prediction via Mixture of Low-Rank Experts](https://mlsanity.com/paper/2403.17749/) (2024)
- [Automated Search for Resource-Efficient Branched Multi-Task Networks](https://mlsanity.com/paper/2008.10292/) (2020)
- [MulT: An End-to-End Multitask Learning Transformer](https://mlsanity.com/paper/2205.08303/) (2022)
- [Learning Multiple Dense Prediction Tasks from Partially Annotated Data](https://mlsanity.com/paper/2111.14893/) (2021)
- [DenseMTL: Cross-task Attention Mechanism for Dense Multi-task Learning](https://mlsanity.com/paper/2206.08927/) (2022)

## Abstract (arXiv)

Multi-task dense prediction, which aims to jointly solve tasks like semantic segmentation and depth estimation, is crucial for robotics applications but suffers from domain shift when deploying models in new environments. While unsupervised domain adaptation (UDA) addresses this challenge for single tasks, existing multi-task UDA methods primarily rely on adversarial learning approaches that are less effective than recent self-training techniques. In this paper, we introduce FAMDA, a simple yet effective UDA framework that addresses this limitation by leveraging Vision Foundation Models (VFMs) as powerful teachers within a self-training paradigm. Our approach integrates Segmentation and Depth foundation models into a self-training paradigm to generate high-quality pseudo-labels for the target domain, effectively distilling their robust generalization capabilities into a single, efficient student network. Extensive experiments show that FAMDA achieves state-of-the-art (SOTA) performance on standard synthetic-to-real UDA multi-task learning (MTL) benchmarks and a challenging new day-to-night adaptation task. Our framework enables the training of highly efficient models; a lightweight variant achieves SOTA accuracy while being more than 10X smaller than foundation models, highlighting FAMDA's suitability for creating domain-adaptive and efficient models for resource-constrained robotics applications.