mlsanityData to 14 Oct 2026

Algorithm-Dependent Bounds for Representation Learning of Multi-Source Domain Adaptation

arXiv 2304.02064 ↗First posted 4 Apr 2023cs.LG stat.MLcited by 7

Not on the map. Why?

Abstract (arXiv)

We use information-theoretic tools to derive a novel analysis of Multi-source Domain Adaptation (MDA) from the representation learning perspective. Concretely, we study joint distribution alignment for supervised MDA with few target labels and unsupervised MDA with pseudo labels, where the latter is relatively hard and less commonly studied. We further provide algorithm-dependent generalization bounds for these two settings, where the generalization is characterized by the mutual information between the parameters and the data. Then we propose a novel deep MDA algorithm, implicitly addressing the target shift through joint alignment. Finally, the mutual information bounds are extended to this algorithm providing a non-vacuous gradient-norm estimation. The proposed algorithm has comparable performance to the state-of-the-art on target-shifted MDA benchmark with improved memory efficiency.

Key facts. Cited by 7 papers in the map (data through 14 Oct 2026). Builds on 34 of its 61 references that are in the map. First posted on arXiv 4 Apr 2023.

Builds on

34 of 61 references.

References · 34

  1. 2015Domain-Adversarial Training of Neural Networkscited by 3,044
  2. 2017Improved Training of Wasserstein GANscited by 2,235
  3. 2016Equality of Opportunity in Supervised Learningcited by 2,007
  4. 2014Unsupervised Domain Adaptation by Backpropagationcited by 1,917
  5. 2018Moment Matching for Multi-Source Domain Adaptationcited by 1,331
  6. 2017Adversarial Discriminative Domain Adaptationcited by 1,311
  7. 2018Computational Optimal Transportcited by 780
  8. 2017Wasserstein GANcited by 639
  9. 2018ADVENT: Adversarial Entropy Minimization for Domain Adaptation in Semantic Segmentationcited by 623
  10. 2022Continual Test-Time Domain Adaptationcited by 525
  11. 2018Detecting and Correcting for Label Shift with Black Box Predictorscited by 366
  12. 2019Semi-supervised Domain Adaptation via Minimax Entropycited by 335
22 more
  1. 2017Information-theoretic analysis of generalization capability of learning algorithmscited by 303
  2. 2017Joint Distribution Optimal Transportation for Domain Adaptationcited by 282
  3. 2020Fairness in Machine Learningcited by 194
  4. 2020Understanding Self-Training for Gradual Domain Adaptationcited by 149
  5. 2019Inherent Tradeoffs in Learning Fair Representationscited by 144
  6. 2020Domain Adaptation with Conditional Distribution Matching and Generalized Label Shiftcited by 125
  7. 2015How much does your data exploration overfit? Controlling bias via information usagecited by 95
  8. 2018Generalization Error Bounds for Noisy, Iterative Algorithmscited by 82
  9. 2018Optimal Transport for Multi-source Domain Adaptation under Target Shiftcited by 68
  10. 2021f-Domain-Adversarial Learning: Theory and Algorithmscited by 47
  11. 2021Generalization Bounds For Meta-Learning: An Information-Theoretic Analysiscited by 45
  12. 2019Domain Aggregation Networks for Multi-Source Domain Adaptationcited by 37
  13. 2019Transfer of Machine Learning Fairness across Domainscited by 36
  14. 2020Information-Theoretic Generalization Bounds for Meta-Learning and Applicationscited by 32
  15. 2022On Learning Fairness and Accuracy on Multiple Subgroupscited by 32
  16. 2013Generalization Bounds for Domain Adaptationcited by 27
  17. 2019Robust Learning from Untrusted Sourcescited by 27
  18. 2021Aggregating From Multiple Target-Shifted Sourcescited by 24
  19. 2020Information-theoretic analysis for transfer learningcited by 23
  20. 2022Fair Representation Learning through Implicit Path Alignmentcited by 22
  21. 2020Beyond H-Divergence: Domain Adaptation Theory With Jensen-Shannon Divergencecited by 15
  22. 2022Gap Minimization for Knowledge Sharing and Transfercited by 4

Built on by

Cited by year

Citing papers per year: 2023: 2; 2024: 3; 2025: 2.

7 papers.

  1. 2024More is Better: Deep Domain Adaptation with Multiple Sourcescited by 9
  2. 2023H-ensemble: An Information Theoretic Approach to Reliable Few-Shot Multi-Source-Free Transfercited by 6
  3. 2025Generalization in VAE and Diffusion Models: A Unified Information-Theoretic Analysiscited by 3
  4. 2024Bridging Domains with Approximately Shared Featurescited by 2
  5. 2024Leveraging Group Classification with Descending Soft Labeling for Deep Imbalanced Regressioncited by 1
  6. 2023Convergences for Minimax Optimization Problems over Infinite-Dimensional Spaces Towards Stability in Adversarial Trainingcited by 0
  7. 2025General and Estimable Learning Bound Unifying Covariate and Concept Shiftscited by 0