# Multi-Source Domain Adaptation with Mixture of Experts

arXiv: https://arxiv.org/abs/1809.02256 · HTML: https://mlsanity.com/paper/1809.02256/

## Key facts

- Cited by 62 papers in the map (data through 14 Oct 2026).
- Placed in MoE large language models, one of 12 directions in Mixture-of-experts.
- Builds on 14 of its 39 references that are in the map.
- First posted on arXiv 7 Sep 2018.

## Placed in

- Mixture-of-experts / [MoE large language models](https://mlsanity.com/moe/moe-large-language-models/)

## Builds on (most cited first, 14 of 14)

- [Outrageously Large Neural Networks: The Sparsely-Gated Mixture-of-Experts Layer](https://mlsanity.com/paper/1701.06538/) (2017)
- [Domain-Adversarial Training of Neural Networks](https://mlsanity.com/paper/1505.07818/) (2015)
- [Prototypical Networks for Few-shot Learning](https://mlsanity.com/paper/1703.05175/) (2017)
- [Matching Networks for One Shot Learning](https://mlsanity.com/paper/1606.04080/) (2016)
- [Convolutional Neural Networks for Sentence Classification](https://mlsanity.com/paper/1408.5882/) (2014)
- [Deep Domain Confusion: Maximizing for Domain Invariance](https://mlsanity.com/paper/1412.3474/) (2014)
- [Domain Separation Networks](https://mlsanity.com/paper/1608.06019/) (2016)
- [Learning to select data for transfer learning with Bayesian Optimization](https://mlsanity.com/paper/1707.05246/) (2017)
- [Diverse Few-Shot Text Classification with Multiple Metrics](https://mlsanity.com/paper/1805.07513/) (2018)
- [Multinomial Adversarial Networks for Multi-Domain Text Classification](https://mlsanity.com/paper/1802.05694/) (2018)
- [Neural Structural Correspondence Learning for Domain Adaptation](https://mlsanity.com/paper/1610.01588/) (2016)
- [Parsing Tweets into Universal Dependencies](https://mlsanity.com/paper/1804.08228/) (2018)
- [Knowledge Adaptation: Teaching to Adapt](https://mlsanity.com/paper/1702.02052/) (2017)
- [Aspect-augmented Adversarial Networks for Domain Adaptation](https://mlsanity.com/paper/1701.00188/) (2017)

## Built on by (most cited first, 30 of 62)

- [BioMistral: A Collection of Open-Source Pretrained Large Language Models for Medical Domains](https://mlsanity.com/paper/2402.10373/) (2024)
- [DEMix Layers: Disentangling Domains for Modular Language Modeling](https://mlsanity.com/paper/2108.05036/) (2021)
- [A Comprehensive Survey of Mixture-of-Experts: Algorithms, Theory, and Applications](https://mlsanity.com/paper/2503.07137/) (2025)
- [Sparse Mixture-of-Experts are Domain Generalizable Learners](https://mlsanity.com/paper/2206.04046/) (2022)
- [Modular Deep Learning](https://mlsanity.com/paper/2302.11529/) (2023)
- [Generalizable Person Re-identification with Relevance-aware Mixture of Experts](https://mlsanity.com/paper/2105.09156/) (2021)
- [Unsupervised Multi-source Domain Adaptation Without Access to Source Data](https://mlsanity.com/paper/2104.01845/) (2021)
- [A Survey on Stance Detection for Mis- and Disinformation Identification](https://mlsanity.com/paper/2103.00242/) (2021)
- [PADA: Example-based Prompt Learning for on-the-fly Adaptation to Unseen Domains](https://mlsanity.com/paper/2102.12206/) (2021)
- [Your Classifier can Secretly Suffice Multi-Source Domain Adaptation](https://mlsanity.com/paper/2103.11169/) (2021)
- [UDALM: Unsupervised Domain Adaptation through Language Modeling](https://mlsanity.com/paper/2104.07078/) (2021)
- [AdaTag: Multi-Attribute Value Extraction from Product Profiles with Adaptive Decoding](https://mlsanity.com/paper/2106.02318/) (2021)
- [Cross-Domain Label-Adaptive Stance Detection](https://mlsanity.com/paper/2104.07467/) (2021)
- [Adaptive Mixture of Experts Learning for Generalizable Face Anti-Spoofing](https://mlsanity.com/paper/2207.09868/) (2022)
- [MoE-FFD: Mixture of Experts for Generalized and Parameter-Efficient Face Forgery Detection](https://mlsanity.com/paper/2404.08452/) (2024)
- [T-SVDNet: Exploring High-Order Prototypical Correlations for Multi-Source Domain Adaptation](https://mlsanity.com/paper/2107.14447/) (2021)
- [CALDA: Improving Multi-Source Time Series Domain Adaptation with Contrastive Adversarial Learning](https://mlsanity.com/paper/2109.14778/) (2021)
- [FastSurferVINN: Building Resolution-Independence into Deep Learning Segmentation Methods -- A Solution for HighRes Brain MRI](https://mlsanity.com/paper/2112.09654/) (2021)
- [On Balancing Bias and Variance in Unsupervised Multi-Source-Free Domain Adaptation](https://mlsanity.com/paper/2202.00796/) (2022)
- [Example-based Hypernetworks for Out-of-Distribution Generalization](https://mlsanity.com/paper/2203.14276/) (2022)
- [A Data Fusion Framework for Multi-Domain Morality Learning](https://mlsanity.com/paper/2304.02144/) (2023)
- [Secure Domain Adaptation with Multiple Sources](https://mlsanity.com/paper/2106.12124/) (2021)
- [Multi-Source domain adaptation via supervised contrastive learning and confident consistency regularization](https://mlsanity.com/paper/2106.16093/) (2021)
- [NeuraCrypt: Hiding Private Health Data via Random Neural Networks for Public Training](https://mlsanity.com/paper/2106.02484/) (2021)
- [Diverse Distributions of Self-Supervised Tasks for Meta-Learning in NLP](https://mlsanity.com/paper/2111.01322/) (2021)
- [DAME: Domain Adaptation for Matching Entities](https://mlsanity.com/paper/2204.09244/) (2022)
- [More is Better: Deep Domain Adaptation with Multiple Sources](https://mlsanity.com/paper/2405.00749/) (2024)
- [HMOE: Hypernetwork-based Mixture of Experts for Domain Generalization](https://mlsanity.com/paper/2211.08253/) (2022)
- [LFME: A Simple Framework for Learning from Multiple Experts in Domain Generalization](https://mlsanity.com/paper/2410.17020/) (2024)
- [Iterative Network Pruning with Uncertainty Regularization for Lifelong Sentiment Classification](https://mlsanity.com/paper/2106.11197/) (2021)

## Abstract (arXiv)

We propose a mixture-of-experts approach for unsupervised domain adaptation from multiple sources. The key idea is to explicitly capture the relationship between a target example and different source domains. This relationship, expressed by a point-to-set metric, determines how to combine predictors trained on various domains. The metric is learned in an unsupervised fashion using meta-training. Experimental results on sentiment analysis and part-of-speech tagging demonstrate that our approach consistently outperforms multiple baselines and can robustly handle negative transfer.