# Maximizing Information in Domain-Invariant Representation Improves Transfer Learning

arXiv: https://arxiv.org/abs/2306.00262 · HTML: https://mlsanity.com/paper/2306.00262/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 26 of its 47 references that are in the map.
- First posted on arXiv 1 Jun 2023.

## Builds on (most cited first, 26 of 26)

- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shift](https://mlsanity.com/paper/1502.03167/) (2015)
- [Domain-Adversarial Training of Neural Networks](https://mlsanity.com/paper/1505.07818/) (2015)
- [Learning Transferable Features with Deep Adaptation Networks](https://mlsanity.com/paper/1502.02791/) (2015)
- [Adversarial Discriminative Domain Adaptation](https://mlsanity.com/paper/1702.05464/) (2017)
- [Deep Hashing Network for Unsupervised Domain Adaptation](https://mlsanity.com/paper/1706.07522/) (2017)
- [A Comprehensive Survey on Transfer Learning](https://mlsanity.com/paper/1911.02685/) (2019)
- [Maximum Classifier Discrepancy for Unsupervised Domain Adaptation](https://mlsanity.com/paper/1712.02560/) (2017)
- [On Lazy Training in Differentiable Programming](https://mlsanity.com/paper/1812.07956/) (2018)
- [Deep Visual Domain Adaptation: A Survey](https://mlsanity.com/paper/1802.03601/) (2018)
- [Domain Separation Networks](https://mlsanity.com/paper/1608.06019/) (2016)
- [Multi-Adversarial Domain Adaptation](https://mlsanity.com/paper/1809.02176/) (2018)
- [Generate To Adapt: Aligning Domains using Generative Adversarial Networks](https://mlsanity.com/paper/1704.01705/) (2017)
- [Domain Agnostic Learning with Disentangled Representations](https://mlsanity.com/paper/1904.12347/) (2019)
- [Deep Unsupervised Domain Adaptation: A Review of Recent Advances and Perspectives](https://mlsanity.com/paper/2208.07422/) (2022)
- [Unsupervised Domain Adaptation with Similarity Learning](https://mlsanity.com/paper/1711.08995/) (2017)
- [Adversarial-Learned Loss for Domain Adaptation](https://mlsanity.com/paper/2001.01046/) (2020)
- [Learning Disentangled Semantic Representation for Domain Adaptation](https://mlsanity.com/paper/2012.11807/) (2020)
- [Transfer Adaptation Learning: A Decade Survey](https://mlsanity.com/paper/1903.04687/) (2019)
- [Spectral Unsupervised Domain Adaptation for Visual Recognition](https://mlsanity.com/paper/2106.06112/) (2021)
- [A Survey of Unsupervised Domain Adaptation for Visual Recognition](https://mlsanity.com/paper/2112.06745/) (2021)
- [Variational Interaction Information Maximization for Cross-domain Disentanglement](https://mlsanity.com/paper/2012.04251/) (2020)
- [Variational Disentanglement for Domain Generalization](https://mlsanity.com/paper/2109.05826/) (2021)
- [Information-theoretic regularization for Multi-source Domain Adaptation](https://mlsanity.com/paper/2104.01568/) (2021)
- [Unsupervised Visual Domain Adaptation: A Deep Max-Margin Gaussian Process Approach](https://mlsanity.com/paper/1902.08727/) (2019)
- [Improved Techniques for Adversarial Discriminative Domain Adaptation](https://mlsanity.com/paper/1809.03625/) (2018)

## Abstract (arXiv)

We propose MaxDIRep, a domain adaptation method that improves the decomposition of data representations into domain-independent and domain-dependent components. Existing methods, such as Domain-Separation Networks (DSN), use a weak orthogonality constraint between these components, which can lead to label-relevant features being partially encoded in the domain-dependent representation (DDRep) rather than the domain-independent representation (DIRep). As a result, information crucial for target-domain classification may be missing from the DIRep. MaxDIRep addresses this issue by applying a Kullback-Leibler (KL) divergence constraint to minimize the information content of the DDRep, thereby encouraging the DIRep to retain features that are both domain-invariant and predictive of target labels. Through geometric analysis and an ablation study on synthetic datasets, we show why DSN's weaker constraint can lead to suboptimal adaptation. Experiments on standard image benchmarks and a network intrusion detection task demonstrate that MaxDIRep achieves strong performance, works with pretrained models, and generalizes to non-image classification tasks.