# Learning Invariant Representations and Risks for Semi-supervised Domain Adaptation

arXiv: https://arxiv.org/abs/2010.04647 · HTML: https://mlsanity.com/paper/2010.04647/

## Key facts

- Cited by 69 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 9 Oct 2020.

## Built on by (most cited first, 30 of 69)

- [Domain Generalization: A Survey](https://mlsanity.com/paper/2103.02503/) (2021)
- [Fishr: Invariant Gradient Variances for Out-of-Distribution Generalization](https://mlsanity.com/paper/2109.02934/) (2021)
- [Invariant Information Bottleneck for Domain Generalization](https://mlsanity.com/paper/2106.06333/) (2021)
- [Edge-Cloud Polarization and Collaboration: A Comprehensive Survey for AI](https://mlsanity.com/paper/2111.06061/) (2021)
- [Environment-Aware Dynamic Graph Learning for Out-of-Distribution Generalization](https://mlsanity.com/paper/2311.11114/) (2023)
- [Federated Domain Generalization: A Survey](https://mlsanity.com/paper/2306.01334/) (2023)
- [Learning to Generalize across Domains on Single Test Samples](https://mlsanity.com/paper/2202.08045/) (2022)
- [On Learning Contrastive Representations for Learning with Noisy Labels](https://mlsanity.com/paper/2203.01785/) (2022)
- [Prior Knowledge Guided Unsupervised Domain Adaptation](https://mlsanity.com/paper/2207.08877/) (2022)
- [On the Effectiveness of Out-of-Distribution Data in Self-Supervised Long-Tail Learning](https://mlsanity.com/paper/2306.04934/) (2023)
- [Surprisingly Simple Semi-Supervised Domain Adaptation with Pretraining and Consistency](https://mlsanity.com/paper/2101.12727/) (2021)
- [Transferable Time-Series Forecasting under Causal Conditional Shift](https://mlsanity.com/paper/2111.03422/) (2021)
- [Learning Distinctive Margin toward Active Domain Adaptation](https://mlsanity.com/paper/2203.05738/) (2022)
- [MentorGNN: Deriving Curriculum for Pre-Training GNNs](https://mlsanity.com/paper/2208.09905/) (2022)
- [Gradual Domain Adaptation: Theory and Algorithms](https://mlsanity.com/paper/2310.13852/) (2023)
- [Convolutional Visual Prompt for Robust Visual Perception](https://mlsanity.com/paper/2303.00198/) (2023)
- [Generalizing across Temporal Domains with Koopman Operators](https://mlsanity.com/paper/2402.07834/) (2024)
- [AdaPose: Towards Cross-Site Device-Free Human Pose Estimation with Commodity WiFi](https://mlsanity.com/paper/2309.16964/) (2023)
- [Foresee What You Will Learn: Data Augmentation for Domain Generalization in Non-stationary Environment](https://mlsanity.com/paper/2301.07845/) (2023)
- [Adaptive Betweenness Clustering for Semi-Supervised Domain Adaptation](https://mlsanity.com/paper/2401.11448/) (2024)
- [Rethinking model prototyping through the MedMNIST+ dataset collection](https://mlsanity.com/paper/2404.15786/) (2024)
- [Test-time Batch Normalization](https://mlsanity.com/paper/2205.10210/) (2022)
- [Learning Unbiased Transferability for Domain Adaptation by Uncertainty Modeling](https://mlsanity.com/paper/2206.01319/) (2022)
- [Domain Generalization via Contrastive Causal Learning](https://mlsanity.com/paper/2210.02655/) (2022)
- [PGrad: Learning Principal Gradients For Domain Generalization](https://mlsanity.com/paper/2305.01134/) (2023)
- [Semi-supervised Drifted Stream Learning with Short Lookback](https://mlsanity.com/paper/2205.13066/) (2022)
- [Text Descriptions are Compressive and Invariant Representations for Visual Learning](https://mlsanity.com/paper/2307.04317/) (2023)
- [Learning Invariant Representation with Consistency and Diversity for Semi-supervised Source Hypothesis Transfer](https://mlsanity.com/paper/2107.03008/) (2021)
- [Multi-layer Representation Learning for Robust OOD Image Classification](https://mlsanity.com/paper/2207.13678/) (2022)
- [Cross-hospital Sepsis Early Detection via Semi-supervised Optimal Transport with Self-paced Ensemble](https://mlsanity.com/paper/2106.10352/) (2021)

## Abstract (arXiv)

The success of supervised learning hinges on the assumption that the training and test data come from the same underlying distribution, which is often not valid in practice due to potential distribution shift. In light of this, most existing methods for unsupervised domain adaptation focus on achieving domain-invariant representations and small source domain error. However, recent works have shown that this is not sufficient to guarantee good generalization on the target domain, and in fact, is provably detrimental under label distribution shift. Furthermore, in many real-world applications it is often feasible to obtain a small amount of labeled data from the target domain and use them to facilitate model training with source data. Inspired by the above observations, in this paper we propose the first method that aims to simultaneously learn invariant representations and risks under the setting of semi-supervised domain adaptation (Semi-DA). First, we provide a finite sample bound for both classification and regression problems under Semi-DA. The bound suggests a principled way to obtain target generalization, i.e. by aligning both the marginal and conditional distributions across domains in feature space. Motivated by this, we then introduce the LIRR algorithm for jointly Learning Invariant Representations and Risks. Finally, extensive experiments are conducted on both classification and regression tasks, which demonstrates LIRR consistently achieves state-of-the-art performance and significant improvements compared with the methods that only learn invariant representations or invariant risks.