# Multi-source Distilling Domain Adaptation

arXiv: https://arxiv.org/abs/1911.11554 · HTML: https://mlsanity.com/paper/1911.11554/

## Key facts

- Cited by 73 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 22 Nov 2019.

## Built on by (most cited first, 30 of 73)

- [Domain Adaptation for Medical Image Analysis: A Survey](https://mlsanity.com/paper/2102.09508/) (2021)
- [Source-Free Domain Adaptation for Semantic Segmentation](https://mlsanity.com/paper/2103.16372/) (2021)
- [MixStyle Neural Networks for Domain Generalization and Adaptation](https://mlsanity.com/paper/2107.02053/) (2021)
- [Meta-DMoE: Adapting to Domain Shift by Meta-Distillation from Mixture-of-Experts](https://mlsanity.com/paper/2210.03885/) (2022)
- [Domain-Agnostic Mutual Prompting for Unsupervised Domain Adaptation](https://mlsanity.com/paper/2403.02899/) (2024)
- [Multi-Source Unsupervised Domain Adaptation via Pseudo Target Domain](https://mlsanity.com/paper/2202.10725/) (2022)
- [Semantic Image Segmentation: Two Decades of Research](https://mlsanity.com/paper/2302.06378/) (2023)
- [Multi-Prompt Alignment for Multi-Source Unsupervised Domain Adaptation](https://mlsanity.com/paper/2209.15210/) (2022)
- [Aggregating From Multiple Target-Shifted Sources](https://mlsanity.com/paper/2105.04051/) (2021)
- [T-SVDNet: Exploring High-Order Prototypical Correlations for Multi-Source Domain Adaptation](https://mlsanity.com/paper/2107.14447/) (2021)
- [Towards Effective Clustered Federated Learning: A Peer-to-peer Framework with Adaptive Neighbor Matching](https://mlsanity.com/paper/2203.12285/) (2022)
- [MADG: Margin-based Adversarial Learning for Domain Generalization](https://mlsanity.com/paper/2311.08503/) (2023)
- [Multi-Source Domain Adaptation for Object Detection](https://mlsanity.com/paper/2106.15793/) (2021)
- [Multi-Source Video Domain Adaptation with Temporal Attentive Moment Alignment](https://mlsanity.com/paper/2109.09964/) (2021)
- [Unsupervised Domain Adaptation for Semantic Image Segmentation: a Comprehensive Survey](https://mlsanity.com/paper/2112.03241/) (2021)
- [On Balancing Bias and Variance in Unsupervised Multi-Source-Free Domain Adaptation](https://mlsanity.com/paper/2202.00796/) (2022)
- [Example-based Hypernetworks for Out-of-Distribution Generalization](https://mlsanity.com/paper/2203.14276/) (2022)
- [Information-theoretic regularization for Multi-source Domain Adaptation](https://mlsanity.com/paper/2104.01568/) (2021)
- [Unified Group Fairness on Federated Learning](https://mlsanity.com/paper/2111.04986/) (2021)
- [On Learning Domain-Invariant Representations for Transfer Learning with Multiple Sources](https://mlsanity.com/paper/2111.13822/) (2021)
- [Transferrable Contrastive Learning for Visual Domain Adaptation](https://mlsanity.com/paper/2112.07516/) (2021)
- [Target-Relevant Knowledge Preservation for Multi-Source Domain Adaptive Object Detection](https://mlsanity.com/paper/2204.07964/) (2022)
- [LanDA: Language-Guided Multi-Source Domain Adaptation](https://mlsanity.com/paper/2401.14148/) (2024)
- [Cross-Modal and Uncertainty-Aware Agglomeration for Open-Vocabulary 3D Scene Understanding](https://mlsanity.com/paper/2503.16707/) (2025)
- [Secure Domain Adaptation with Multiple Sources](https://mlsanity.com/paper/2106.12124/) (2021)
- [Multi-Source domain adaptation via supervised contrastive learning and confident consistency regularization](https://mlsanity.com/paper/2106.16093/) (2021)
- [Towards Data-Free Domain Generalization](https://mlsanity.com/paper/2110.04545/) (2021)
- [Subject-Based Domain Adaptation for Facial Expression Recognition](https://mlsanity.com/paper/2312.05632/) (2023)
- [More is Better: Deep Domain Adaptation with Multiple Sources](https://mlsanity.com/paper/2405.00749/) (2024)
- [Direct Distillation between Different Domains](https://mlsanity.com/paper/2401.06826/) (2024)

## Abstract (arXiv)

Deep neural networks suffer from performance decay when there is domain shift between the labeled source domain and unlabeled target domain, which motivates the research on domain adaptation (DA). Conventional DA methods usually assume that the labeled data is sampled from a single source distribution. However, in practice, labeled data may be collected from multiple sources, while naive application of the single-source DA algorithms may lead to suboptimal solutions. In this paper, we propose a novel multi-source distilling domain adaptation (MDDA) network, which not only considers the different distances among multiple sources and the target, but also investigates the different similarities of the source samples to the target ones. Specifically, the proposed MDDA includes four stages: (1) pre-train the source classifiers separately using the training data from each source; (2) adversarially map the target into the feature space of each source respectively by minimizing the empirical Wasserstein distance between source and target; (3) select the source training samples that are closer to the target to fine-tune the source classifiers; and (4) classify each encoded target feature by corresponding source classifier, and aggregate different predictions using respective domain weight, which corresponds to the discrepancy between each source and target. Extensive experiments are conducted on public DA benchmarks, and the results demonstrate that the proposed MDDA significantly outperforms the state-of-the-art approaches. Our source code is released at: https://github.com/daoyuan98/MDDA.