# Trade-off between reconstruction loss and feature alignment for domain generalization

arXiv: https://arxiv.org/abs/2210.15000 · HTML: https://mlsanity.com/paper/2210.15000/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Builds on 30 of its 40 references that are in the map.
- First posted on arXiv 26 Oct 2022.

## Builds on (most cited first, 30 of 30)

- [Domain-Adversarial Training of Neural Networks](https://mlsanity.com/paper/1505.07818/) (2015)
- [Invariant Risk Minimization](https://mlsanity.com/paper/1907.02893/) (2019)
- [Deep CORAL: Correlation Alignment for Deep Domain Adaptation](https://mlsanity.com/paper/1607.01719/) (2016)
- [In Search of Lost Domain Generalization](https://mlsanity.com/paper/2007.01434/) (2020)
- [Generalizing to Unseen Domains: A Survey on Domain Generalization](https://mlsanity.com/paper/2103.03097/) (2021)
- [Domain Generalization: A Survey](https://mlsanity.com/paper/2103.02503/) (2021)
- [Domain Generalization via Invariant Feature Representation](https://mlsanity.com/paper/1301.2115/) (2013)
- [The Risks of Invariant Risk Minimization](https://mlsanity.com/paper/2010.05761/) (2020)
- [Domain Generalization using Causal Matching](https://mlsanity.com/paper/2006.07500/) (2020)
- [Invariance Principle Meets Information Bottleneck for Out-of-Distribution Generalization](https://mlsanity.com/paper/2106.06607/) (2021)
- [Invariant Risk Minimization Games](https://mlsanity.com/paper/2002.04692/) (2020)
- [Domain Generalization by Marginal Transfer Learning](https://mlsanity.com/paper/1711.07910/) (2017)
- [Understanding the Failure Modes of Out-of-Distribution Generalization](https://mlsanity.com/paper/2010.15775/) (2020)
- [Does Invariant Risk Minimization Capture Invariance?](https://mlsanity.com/paper/2101.01134/) (2021)
- [Learning to Learn with Variational Information Bottleneck for Domain Generalization](https://mlsanity.com/paper/2007.07645/) (2020)
- [Scatter Component Analysis: A Unified Framework for Domain Adaptation and Domain Generalization](https://mlsanity.com/paper/1510.04373/) (2015)
- [Support and Invertibility in Domain-Invariant Representations](https://mlsanity.com/paper/1903.03448/) (2019)
- [Invariant Information Bottleneck for Domain Generalization](https://mlsanity.com/paper/2106.06333/) (2021)
- [Exploiting Domain-Specific Features to Enhance Domain Generalization](https://mlsanity.com/paper/2110.09410/) (2021)
- [Empirical or Invariant Risk Minimization? A Sample Complexity Perspective](https://mlsanity.com/paper/2010.16412/) (2020)
- [From source to target and back: symmetric bi-directional adaptive GAN](https://mlsanity.com/paper/1705.08824/) (2017)
- [Nonlinear Invariant Risk Minimization: A Causal Approach](https://mlsanity.com/paper/2102.12353/) (2021)
- [Feature Alignment and Restoration for Domain Generalization and Adaptation](https://mlsanity.com/paper/2006.12009/) (2020)
- [Respecting Domain Relations: Hypothesis Invariance for Domain Generalization](https://mlsanity.com/paper/2010.07591/) (2020)
- [Iterative Feature Matching: Toward Provable Domain Generalization with Logarithmic Environments](https://mlsanity.com/paper/2106.09913/) (2021)
- [Out-of-distribution Prediction with Invariant Risk Minimization: The Limitation and An Effective Fix](https://mlsanity.com/paper/2101.07732/) (2021)
- [Linear unit-tests for invariance discovery](https://mlsanity.com/paper/2102.10867/) (2021)
- [Conditional entropy minimization principle for learning domain invariant representation features](https://mlsanity.com/paper/2201.10460/) (2022)
- [Class Distribution Alignment for Adversarial Domain Adaptation](https://mlsanity.com/paper/2004.09403/) (2020)
- [Joint covariate-alignment and concept-alignment: a framework for domain generalization](https://mlsanity.com/paper/2208.00898/) (2022)

## Built on by (most cited first, 1 of 1)

- [WildIng: A Wildlife Image Invariant Representation Model for Geographical Domain Shift](https://mlsanity.com/paper/2601.00993/) (2026)

## Abstract (arXiv)

Domain generalization (DG) is a branch of transfer learning that aims to train the learning models on several seen domains and subsequently apply these pre-trained models to other unseen (unknown but related) domains. To deal with challenging settings in DG where both data and label of the unseen domain are not available at training time, the most common approach is to design the classifiers based on the domain-invariant representation features, i.e., the latent representations that are unchanged and transferable between domains. Contrary to popular belief, we show that designing classifiers based on invariant representation features alone is necessary but insufficient in DG. Our analysis indicates the necessity of imposing a constraint on the reconstruction loss induced by representation functions to preserve most of the relevant information about the label in the latent space. More importantly, we point out the trade-off between minimizing the reconstruction loss and achieving domain alignment in DG. Our theoretical results motivate a new DG framework that jointly optimizes the reconstruction loss and the domain discrepancy. Both theoretical and numerical results are provided to justify our approach.