mlsanityData to 14 Oct 2026

Information-Theoretic Analysis of Unsupervised Domain Adaptation

arXiv 2210.00706 ↗First posted 3 Oct 2022cs.LGcited by 15

Not on the map. Why?

Abstract (arXiv)

This paper uses information-theoretic tools to analyze the generalization error in unsupervised domain adaptation (UDA). We present novel upper bounds for two notions of generalization errors. The first notion measures the gap between the population risk in the target domain and that in the source domain, and the second measures the gap between the population risk in the target domain and the empirical risk in the source domain. While our bounds for the first kind of error are in line with the traditional analysis and give similar insights, our bounds on the second kind of error are algorithm-dependent, which also provide insights into algorithm designs. Specifically, we present two simple techniques for improving generalization in UDA and validate them experimentally.

Key facts. Cited by 15 papers in the map (data through 14 Oct 2026). Builds on 44 of its 90 references that are in the map. First posted on arXiv 3 Oct 2022.

Builds on

44 of 90 references.

References · 44

  1. 2019PyTorch: An Imperative Style, High-Performance Deep Learning Librarycited by 20,305
  2. 2015Domain-Adversarial Training of Neural Networkscited by 3,044
  3. 2017Improved Training of Wasserstein GANscited by 2,235
  4. 2016Deep CORAL: Correlation Alignment for Deep Domain Adaptationcited by 1,200
  5. 2020In Search of Lost Domain Generalizationcited by 1,054
  6. 2020Do We Really Need to Access the Source Data? Source Hypothesis Transfer for Unsupervised Domain Adaptationcited by 848
  7. 2021Generalizing to Unseen Domains: A Survey on Domain Generalizationcited by 766
  8. 2021Domain Generalization: A Surveycited by 764
  9. 2017VisDA: The Visual Domain Adaptation Challengecited by 435
  10. 2018A Survey of Unsupervised Deep Domain Adaptationcited by 361
  11. 2019Bridging Theory and Algorithm for Domain Adaptationcited by 332
  12. 2017Information-theoretic analysis of generalization capability of learning algorithmscited by 303
32 more
  1. 2017Emergence of Invariance and Disentanglement in Deep Representationscited by 215
  2. 2021On the Origin of Implicit Regularization in Stochastic Gradient Descentcited by 184
  3. 2020Reasoning About Generalization via Conditional Mutual Informationcited by 147
  4. 2017Domain Adaptation for Visual Applications: A Comprehensive Surveycited by 128
  5. 2019Information-Theoretic Generalization Bounds for SGLD via Data-Dependent Estimatescited by 119
  6. 2019Tightening Mutual Information Based Bounds on Generalization Errorcited by 117
  7. 2020A survey on domain adaptation theory: learning bounds and theoretical guaranteescited by 109
  8. 2015How much does your data exploration overfit? Controlling bias via information usagecited by 95
  9. 2020Sharpened Generalization Bounds based on Conditional Mutual Information and an Application to Noisy, Iterative Algorithmscited by 89
  10. 2021Information-Theoretic Generalization Bounds for Stochastic Gradient Descentcited by 84
  11. 2018Generalization Error Bounds for Noisy, Iterative Algorithmscited by 82
  12. 2021Stochastic Training is Not Necessary for Generalizationcited by 63
  13. 2020Catastrophic Fisher Explosion: Early Phase Fisher Matrix Impacts Generalizationcited by 62
  14. 2022Predicting Out-of-Distribution Error with the Projection Normcited by 50
  15. 2021f-Domain-Adversarial Learning: Theory and Algorithmscited by 47
  16. 2021Generalization Bounds For Meta-Learning: An Information-Theoretic Analysiscited by 45
  17. 2021Tighter expected generalization error bounds via Wasserstein distancecited by 43
  18. 2020Improving Generalization by Controlling Label-Noise Information in Neural Network Weightscited by 35
  19. 2021KL Guided Domain Adaptationcited by 35
  20. 2020Information-Theoretic Generalization Bounds for Meta-Learning and Applicationscited by 32
  21. 2021On the Generalization of Models Trained with SGD: Information-Theoretic Bounds and Implicationscited by 31
  22. 2020On Random Subset Generalization Error Bounds and the Stochastic Gradient Langevin Dynamics Algorithmcited by 28
  23. 2020Optimal Bounds between f-Divergences and Integral Probability Metricscited by 27
  24. 2020Information-theoretic analysis for transfer learningcited by 23
  25. 2022An Information-theoretical Approach to Semi-supervised Learning under Covariate-shiftcited by 18
  26. 2017PAC-Bayes and Domain Adaptationcited by 16
  27. 2013Relative Deviation Learning Bounds and Generalization with Unbounded Loss Functionscited by 15
  28. 2020Beyond H-Divergence: Domain Adaptation Theory With Jensen-Shannon Divergencecited by 15
  29. 2021Generalization Bounds for Noisy Iterative Algorithms Using Properties of Additive Noise Channelscited by 15
  30. 2020Transfer Meta-Learning: Information-Theoretic Bounds and Information Meta-Risk Minimizationcited by 14
  31. 2021Characterizing and Understanding the Generalization Error of Transfer Learning with Gibbs Algorithmcited by 14
  32. 2020Information-Theoretic Bounds on Transfer Generalization Gap Based on Jensen-Shannon Divergencecited by 11

Built on by

Cited by year

Citing papers per year: 2023: 4; 2024: 5; 2025: 6.

15 papers.

Sort:

By area:

  1. 2023Tighter Information-Theoretic Generalization Bounds from Supersamplescited by 22
  2. 2024Enhancing Domain Adaptation through Prompt Gradient Alignmentcited by 8
  3. 2023Sample-Conditioned Hypothesis Stability Sharpens Information-Theoretic Generalization Boundscited by 6
  4. 2024On f-Divergence Principled Domain Adaptation: An Improved Frameworkcited by 6
  5. 2025Domain Adaptation and Entanglement: an Optimal Transport Perspectivecited by 6
  6. 2023Out-Of-Domain Unlabeled Data Improves Generalizationcited by 3
  7. 2024An Information-Theoretic Framework for Out-of-Distribution Generalization with Applications to Stochastic Gradient Langevin Dynamicscited by 3
  8. 2024How Does Distribution Matching Help Domain Generalization: An Information-theoretic Analysiscited by 3
  9. 2023Information-Theoretic Generalization Bounds for Transductive Learning and its Applicationscited by 2
  10. 2024Class-wise Generalization Error: an Information-Theoretic Analysiscited by 2
  11. 2025Generalization in Federated Learning: A Conditional Mutual Information Frameworkcited by 2
  12. 2025Adaptive Estimation and Learning under Temporal Distribution Shiftcited by 2
3 more
  1. 2025FedDAA: Dynamic Client Clustering for Concept Drift Adaptation in Federated Learningcited by 1
  2. 2025On the Hardness of Unsupervised Domain Adaptation: Optimal Learners and Information-Theoretic Perspectivecited by 1
  3. 2025An Information-Theoretic Analysis of OOD Generalization in Meta-Reinforcement Learningcited by 0