# UM-Adapt: Unsupervised Multi-Task Adaptation Using Adversarial Cross-Task Distillation

arXiv: https://arxiv.org/abs/1908.03884 · HTML: https://mlsanity.com/paper/1908.03884/

## Key facts

- Cited by 23 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 11 Aug 2019.

## Built on by (most cited first, 23 of 23)

- [Generalize then Adapt: Source-Free Domain Adaptive Semantic Segmentation](https://mlsanity.com/paper/2108.11249/) (2021)
- [Your Classifier can Secretly Suffice Multi-Source Domain Adaptation](https://mlsanity.com/paper/2103.11169/) (2021)
- [Cross-View Regularization for Domain Adaptive Panoptic Segmentation](https://mlsanity.com/paper/2103.02584/) (2021)
- [Multi-Task Learning with Multi-Query Transformer for Dense Prediction](https://mlsanity.com/paper/2205.14354/) (2022)
- [Learning Scene Structure Guidance via Cross-Task Knowledge Transfer for Single Depth Super-Resolution](https://mlsanity.com/paper/2103.12955/) (2021)
- [Uncertainty-Aware Adaptation for Self-Supervised 3D Human Pose Estimation](https://mlsanity.com/paper/2203.15293/) (2022)
- [A Unified Framework for Domain Adaptive Pose Estimation](https://mlsanity.com/paper/2204.00172/) (2022)
- [Divide to Adapt: Mitigating Confirmation Bias for Domain Adaptation of Black-Box Predictors](https://mlsanity.com/paper/2205.14467/) (2022)
- [PanopticPartFormer++: A Unified and Decoupled View for Panoptic Part Segmentation](https://mlsanity.com/paper/2301.00954/) (2023)
- [Plugging Self-Supervised Monocular Depth into Unsupervised Domain Adaptation for Semantic Segmentation](https://mlsanity.com/paper/2110.06685/) (2021)
- [Generalized Knowledge Distillation via Relationship Matching](https://mlsanity.com/paper/2205.01915/) (2022)
- [Amplitude Spectrum Transformation for Open Compound Domain Adaptive Semantic Segmentation](https://mlsanity.com/paper/2202.04287/) (2022)
- [Unsupervised Cross-Modal Distillation for Thermal Infrared Tracking](https://mlsanity.com/paper/2108.00187/) (2021)
- [Non-Local Latent Relation Distillation for Self-Adaptive 3D Human Pose Estimation](https://mlsanity.com/paper/2204.01971/) (2022)
- [Aligning Silhouette Topology for Self-Adaptive 3D Human Pose Recovery](https://mlsanity.com/paper/2204.01276/) (2022)
- [Adaptive Distillation: Aggregating Knowledge from Multiple Paths for Efficient Distillation](https://mlsanity.com/paper/2110.09674/) (2021)
- [Learn to Adapt for Monocular Depth Estimation](https://mlsanity.com/paper/2203.14005/) (2022)
- [Multi-task Geometric Estimation of Depth and Surface Normal from Monocular 360deg Images](https://mlsanity.com/paper/2411.01749/) (2024)
- [Learning Good Features to Transfer Across Tasks and Domains](https://mlsanity.com/paper/2301.11310/) (2023)
- [Domain Adaptive Monocular Depth Estimation With Semantic Information](https://mlsanity.com/paper/2104.05764/) (2021)
- [Boosting Supervised Learning Performance with Co-training](https://mlsanity.com/paper/2111.09797/) (2021)
- [SciceVPR: Stable Cross-Image Correlation Enhanced Model for Visual Place Recognition](https://mlsanity.com/paper/2502.20676/) (2025)
- [Efficient Domain-Adaptive Multi-Task Dense Prediction with Vision Foundation Models](https://mlsanity.com/paper/2509.23626/) (2025)

## Abstract (arXiv)

Aiming towards human-level generalization, there is a need to explore adaptable representation learning methods with greater transferability. Most existing approaches independently address task-transferability and cross-domain adaptation, resulting in limited generalization. In this paper, we propose UM-Adapt - a unified framework to effectively perform unsupervised domain adaptation for spatially-structured prediction tasks, simultaneously maintaining a balanced performance across individual tasks in a multi-task setting. To realize this, we propose two novel regularization strategies; a) Contour-based content regularization (CCR) and b) exploitation of inter-task coherency using a cross-task distillation module. Furthermore, avoiding a conventional ad-hoc domain discriminator, we re-utilize the cross-task distillation loss as output of an energy function to adversarially minimize the input domain discrepancy. Through extensive experiments, we demonstrate superior generalizability of the learned representations simultaneously for multiple tasks under domain-shifts from synthetic to natural environments. UM-Adapt yields state-of-the-art transfer learning results on ImageNet classification and comparable performance on PASCAL VOC 2007 detection task, even with a smaller backbone-net. Moreover, the resulting semi-supervised framework outperforms the current fully-supervised multi-task learning state-of-the-art on both NYUD and Cityscapes dataset.