# Support and Invertibility in Domain-Invariant Representations

arXiv: https://arxiv.org/abs/1903.03448 · HTML: https://mlsanity.com/paper/1903.03448/

## Key facts

- Cited by 87 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 8 Mar 2019.

## Built on by (most cited first, 30 of 87)

- [Generalizing to Unseen Domains: A Survey on Domain Generalization](https://mlsanity.com/paper/2103.03097/) (2021)
- [Fishr: Invariant Gradient Variances for Out-of-Distribution Generalization](https://mlsanity.com/paper/2109.02934/) (2021)
- [A Fine-Grained Analysis on Distribution Shift](https://mlsanity.com/paper/2110.11328/) (2021)
- [Self-supervised Augmentation Consistency for Adapting Semantic Segmentation](https://mlsanity.com/paper/2105.00097/) (2021)
- [Nonparametric Estimation of Heterogeneous Treatment Effects: From Theory to Learning Algorithms](https://mlsanity.com/paper/2101.10943/) (2021)
- [Cycle Self-Training for Domain Adaptation](https://mlsanity.com/paper/2103.03571/) (2021)
- [Towards a Theoretical Framework of Out-of-Distribution Generalization](https://mlsanity.com/paper/2106.04496/) (2021)
- [Invariant Information Bottleneck for Domain Generalization](https://mlsanity.com/paper/2106.06333/) (2021)
- [Exploiting Domain-Specific Features to Enhance Domain Generalization](https://mlsanity.com/paper/2110.09410/) (2021)
- [Desiderata for Representation Learning: A Causal Perspective](https://mlsanity.com/paper/2109.03795/) (2021)
- [A Prototype-Oriented Framework for Unsupervised Domain Adaptation](https://mlsanity.com/paper/2110.12024/) (2021)
- [Continuous-Time Modeling of Counterfactual Outcomes Using Neural Controlled Differential Equations](https://mlsanity.com/paper/2206.08311/) (2022)
- [Causally motivated Shortcut Removal Using Auxiliary Labels](https://mlsanity.com/paper/2105.06422/) (2021)
- [Optimal Representations for Covariate Shift](https://mlsanity.com/paper/2201.00057/) (2021)
- [Domain Adversarial Neural Networks for Domain Generalization: When It Works and How to Improve](https://mlsanity.com/paper/2102.03924/) (2021)
- [KL Guided Domain Adaptation](https://mlsanity.com/paper/2106.07780/) (2021)
- [Modeling the Data-Generating Process is Necessary for Out-of-Distribution Generalization](https://mlsanity.com/paper/2206.07837/) (2022)
- [Domain-invariant Feature Exploration for Domain Generalization](https://mlsanity.com/paper/2207.12020/) (2022)
- [RLSbench: Domain Adaptation Under Relaxed Label Shift](https://mlsanity.com/paper/2302.03020/) (2023)
- [An Information-theoretic Approach to Distribution Shifts](https://mlsanity.com/paper/2106.03783/) (2021)
- [SurvITE: Learning Heterogeneous Treatment Effects from Time-to-Event Data](https://mlsanity.com/paper/2110.14001/) (2021)
- [Aggregating From Multiple Target-Shifted Sources](https://mlsanity.com/paper/2105.04051/) (2021)
- [Out-of-Domain Robustness via Targeted Augmentations](https://mlsanity.com/paper/2302.11861/) (2023)
- [ADAST: Attentive Cross-domain EEG-based Sleep Staging Framework with Iterative Self-Training](https://mlsanity.com/paper/2107.04470/) (2021)
- [On Learning Domain-Invariant Representations for Transfer Learning with Multiple Sources](https://mlsanity.com/paper/2111.13822/) (2021)
- [Limitations of Post-Hoc Feature Alignment for Robustness](https://mlsanity.com/paper/2103.05898/) (2021)
- [A Contrastive Objective for Learning Disentangled Representations](https://mlsanity.com/paper/2203.11284/) (2022)
- [Causal Effect Estimation: Recent Advances, Challenges, and Opportunities](https://mlsanity.com/paper/2302.00848/) (2023)
- [Deep Multi-Modal Structural Equations For Causal Effect Estimation With Unstructured Proxies](https://mlsanity.com/paper/2203.09672/) (2022)
- [PAC-Bayesian Domain Adaptation Bounds for Multiclass Learners](https://mlsanity.com/paper/2207.05685/) (2022)

## Abstract (arXiv)

Learning domain-invariant representations has become a popular approach to unsupervised domain adaptation and is often justified by invoking a particular suite of theoretical results. We argue that there are two significant flaws in such arguments. First, the results in question hold only for a fixed representation and do not account for information lost in non-invertible transformations. Second, domain invariance is often a far too strict requirement and does not always lead to consistent estimation, even under strong and favorable assumptions. In this work, we give generalization bounds for unsupervised domain adaptation that hold for any representation function by acknowledging the cost of non-invertibility. In addition, we show that penalizing distance between densities is often wasteful and propose a bound based on measuring the extent to which the support of the source domain covers the target domain. We perform experiments on well-known benchmarks that illustrate the short-comings of current standard practice.