# Algorithm-Dependent Bounds for Representation Learning of Multi-Source Domain Adaptation

arXiv: https://arxiv.org/abs/2304.02064 · HTML: https://mlsanity.com/paper/2304.02064/

## Key facts

- Cited by 7 papers in the map (data through 14 Oct 2026).
- Builds on 34 of its 61 references that are in the map.
- First posted on arXiv 4 Apr 2023.

## Builds on (most cited first, 30 of 34)

- [Domain-Adversarial Training of Neural Networks](https://mlsanity.com/paper/1505.07818/) (2015)
- [Improved Training of Wasserstein GANs](https://mlsanity.com/paper/1704.00028/) (2017)
- [Equality of Opportunity in Supervised Learning](https://mlsanity.com/paper/1610.02413/) (2016)
- [Unsupervised Domain Adaptation by Backpropagation](https://mlsanity.com/paper/1409.7495/) (2014)
- [Moment Matching for Multi-Source Domain Adaptation](https://mlsanity.com/paper/1812.01754/) (2018)
- [Adversarial Discriminative Domain Adaptation](https://mlsanity.com/paper/1702.05464/) (2017)
- [Computational Optimal Transport](https://mlsanity.com/paper/1803.00567/) (2018)
- [Wasserstein GAN](https://mlsanity.com/paper/1701.07875/) (2017)
- [ADVENT: Adversarial Entropy Minimization for Domain Adaptation in Semantic Segmentation](https://mlsanity.com/paper/1811.12833/) (2018)
- [Continual Test-Time Domain Adaptation](https://mlsanity.com/paper/2203.13591/) (2022)
- [Detecting and Correcting for Label Shift with Black Box Predictors](https://mlsanity.com/paper/1802.03916/) (2018)
- [Semi-supervised Domain Adaptation via Minimax Entropy](https://mlsanity.com/paper/1904.06487/) (2019)
- [Information-theoretic analysis of generalization capability of learning algorithms](https://mlsanity.com/paper/1705.07809/) (2017)
- [Joint Distribution Optimal Transportation for Domain Adaptation](https://mlsanity.com/paper/1705.08848/) (2017)
- [Fairness in Machine Learning](https://mlsanity.com/paper/2012.15816/) (2020)
- [Understanding Self-Training for Gradual Domain Adaptation](https://mlsanity.com/paper/2002.11361/) (2020)
- [Inherent Tradeoffs in Learning Fair Representations](https://mlsanity.com/paper/1906.08386/) (2019)
- [Domain Adaptation with Conditional Distribution Matching and Generalized Label Shift](https://mlsanity.com/paper/2003.04475/) (2020)
- [How much does your data exploration overfit? Controlling bias via information usage](https://mlsanity.com/paper/1511.05219/) (2015)
- [Generalization Error Bounds for Noisy, Iterative Algorithms](https://mlsanity.com/paper/1801.04295/) (2018)
- [Optimal Transport for Multi-source Domain Adaptation under Target Shift](https://mlsanity.com/paper/1803.04899/) (2018)
- [f-Domain-Adversarial Learning: Theory and Algorithms](https://mlsanity.com/paper/2106.11344/) (2021)
- [Generalization Bounds For Meta-Learning: An Information-Theoretic Analysis](https://mlsanity.com/paper/2109.14595/) (2021)
- [Domain Aggregation Networks for Multi-Source Domain Adaptation](https://mlsanity.com/paper/1909.05352/) (2019)
- [Transfer of Machine Learning Fairness across Domains](https://mlsanity.com/paper/1906.09688/) (2019)
- [Information-Theoretic Generalization Bounds for Meta-Learning and Applications](https://mlsanity.com/paper/2005.04372/) (2020)
- [On Learning Fairness and Accuracy on Multiple Subgroups](https://mlsanity.com/paper/2210.10837/) (2022)
- [Generalization Bounds for Domain Adaptation](https://mlsanity.com/paper/1304.1574/) (2013)
- [Robust Learning from Untrusted Sources](https://mlsanity.com/paper/1901.10310/) (2019)
- [Aggregating From Multiple Target-Shifted Sources](https://mlsanity.com/paper/2105.04051/) (2021)

## Built on by (most cited first, 7 of 7)

- [More is Better: Deep Domain Adaptation with Multiple Sources](https://mlsanity.com/paper/2405.00749/) (2024)
- [H-ensemble: An Information Theoretic Approach to Reliable Few-Shot Multi-Source-Free Transfer](https://mlsanity.com/paper/2312.12489/) (2023)
- [Generalization in VAE and Diffusion Models: A Unified Information-Theoretic Analysis](https://mlsanity.com/paper/2506.00849/) (2025)
- [Bridging Domains with Approximately Shared Features](https://mlsanity.com/paper/2403.06424/) (2024)
- [Leveraging Group Classification with Descending Soft Labeling for Deep Imbalanced Regression](https://mlsanity.com/paper/2412.12327/) (2024)
- [Convergences for Minimax Optimization Problems over Infinite-Dimensional Spaces Towards Stability in Adversarial Training](https://mlsanity.com/paper/2312.00991/) (2023)
- [General and Estimable Learning Bound Unifying Covariate and Concept Shifts](https://mlsanity.com/paper/2506.12829/) (2025)

## Abstract (arXiv)

We use information-theoretic tools to derive a novel analysis of Multi-source Domain Adaptation (MDA) from the representation learning perspective. Concretely, we study joint distribution alignment for supervised MDA with few target labels and unsupervised MDA with pseudo labels, where the latter is relatively hard and less commonly studied. We further provide algorithm-dependent generalization bounds for these two settings, where the generalization is characterized by the mutual information between the parameters and the data. Then we propose a novel deep MDA algorithm, implicitly addressing the target shift through joint alignment. Finally, the mutual information bounds are extended to this algorithm providing a non-vacuous gradient-norm estimation. The proposed algorithm has comparable performance to the state-of-the-art on target-shifted MDA benchmark with improved memory efficiency.