# On Learning Invariant Representation for Domain Adaptation

arXiv: https://arxiv.org/abs/1901.09453 · HTML: https://mlsanity.com/paper/1901.09453/

## Key facts

- Cited by 39 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 27 Jan 2019.

## Built on by (most cited first, 30 of 39)

- [Prototypical Pseudo Label Denoising and Target Structure Learning for Domain Adaptive Semantic Segmentation](https://mlsanity.com/paper/2101.10979/) (2021)
- [Invariance Principle Meets Information Bottleneck for Out-of-Distribution Generalization](https://mlsanity.com/paper/2106.06607/) (2021)
- [Exploiting Domain-Specific Features to Enhance Domain Generalization](https://mlsanity.com/paper/2110.09410/) (2021)
- [Connect, Not Collapse: Explaining Contrastive Learning for Unsupervised Domain Adaptation](https://mlsanity.com/paper/2204.00570/) (2022)
- [PromptStyler: Prompt-driven Style Generation for Source-free Domain Generalization](https://mlsanity.com/paper/2307.15199/) (2023)
- [Reconsidering Representation Alignment for Multi-view Clustering](https://mlsanity.com/paper/2103.07738/) (2021)
- [A Theory of Label Propagation for Subpopulation Shift](https://mlsanity.com/paper/2102.11203/) (2021)
- [f-Domain-Adversarial Learning: Theory and Algorithms](https://mlsanity.com/paper/2106.11344/) (2021)
- [Beyond Separability: Analyzing the Linear Transferability of Contrastive Representations to Related Subpopulations](https://mlsanity.com/paper/2204.02683/) (2022)
- [Near-Optimal Linear Regression under Distribution Shift](https://mlsanity.com/paper/2106.12108/) (2021)
- [Modeling the Data-Generating Process is Necessary for Out-of-Distribution Generalization](https://mlsanity.com/paper/2206.07837/) (2022)
- [Semantic Image Segmentation: Two Decades of Research](https://mlsanity.com/paper/2302.06378/) (2023)
- [Assessing Fairness in the Presence of Missing Data](https://mlsanity.com/paper/2112.04899/) (2021)
- [Unsupervised Domain Adaptation for Semantic Image Segmentation: a Comprehensive Survey](https://mlsanity.com/paper/2112.03241/) (2021)
- [Subtype-aware Unsupervised Domain Adaptation for Medical Diagnosis](https://mlsanity.com/paper/2101.00318/) (2021)
- [Information-theoretic regularization for Multi-source Domain Adaptation](https://mlsanity.com/paper/2104.01568/) (2021)
- [Limitations of Post-Hoc Feature Alignment for Robustness](https://mlsanity.com/paper/2103.05898/) (2021)
- [Towards Optimal Strategies for Training Self-Driving Perception Models in Simulation](https://mlsanity.com/paper/2111.07971/) (2021)
- [Gradual Domain Adaptation via Self-Training of Auxiliary Models](https://mlsanity.com/paper/2106.09890/) (2021)
- [Disentanglement and Generalization Under Correlation Shifts](https://mlsanity.com/paper/2112.14754/) (2021)
- [Robust Local Preserving and Global Aligning Network for Adversarial Domain Adaptation](https://mlsanity.com/paper/2203.04156/) (2022)
- [Novelty Detection in Reinforcement Learning with World Models](https://mlsanity.com/paper/2310.08731/) (2023)
- [Modeling Multiple Views via Implicitly Preserving Global Consistency and Local Complementarity](https://mlsanity.com/paper/2209.07811/) (2022)
- [Prominent Roles of Conditionally Invariant Components in Domain Adaptation: Theory and Algorithms](https://mlsanity.com/paper/2309.10301/) (2023)
- [Out-Of-Domain Unlabeled Data Improves Generalization](https://mlsanity.com/paper/2310.00027/) (2023)
- [Subtype-Aware Dynamic Unsupervised Domain Adaptation](https://mlsanity.com/paper/2208.07754/) (2022)
- [Cross-domain error minimization for unsupervised domain adaptation](https://mlsanity.com/paper/2106.15057/) (2021)
- [Information Theory-Guided Heuristic Progressive Multi-View Coding](https://mlsanity.com/paper/2109.02344/) (2021)
- [On the inductive biases of deep domain adaptation](https://mlsanity.com/paper/2109.07920/) (2021)
- [Unsupervised domain adaptation with non-stochastic missing data](https://mlsanity.com/paper/2109.09505/) (2021)

## Abstract (arXiv)

Due to the ability of deep neural nets to learn rich representations, recent advances in unsupervised domain adaptation have focused on learning domain-invariant features that achieve a small error on the source domain. The hope is that the learnt representation, together with the hypothesis learnt from the source domain, can generalize to the target domain. In this paper, we first construct a simple counterexample showing that, contrary to common belief, the above conditions are not sufficient to guarantee successful domain adaptation. In particular, the counterexample exhibits conditional shift: the class-conditional distributions of input features change between source and target domains. To give a sufficient condition for domain adaptation, we propose a natural and interpretable generalization upper bound that explicitly takes into account the aforementioned shift. Moreover, we shed new light on the problem by proving an information-theoretic lower bound on the joint error of any domain adaptation method that attempts to learn invariant representations. Our result characterizes a fundamental tradeoff between learning invariant representations and achieving small joint error on both domains when the marginal label distributions differ from source to target. Finally, we conduct experiments on real-world datasets that corroborate our theoretical findings. We believe these insights are helpful in guiding the future design of domain adaptation and representation learning algorithms.