# On the inductive biases of deep domain adaptation

arXiv: https://arxiv.org/abs/2109.07920 · HTML: https://mlsanity.com/paper/2109.07920/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Builds on 33 of its 47 references that are in the map.
- First posted on arXiv 16 Sep 2021.

## Builds on (most cited first, 30 of 33)

- [Model-Agnostic Meta-Learning for Fast Adaptation of Deep Networks](https://mlsanity.com/paper/1703.03400/) (2017)
- [Domain-Adversarial Training of Neural Networks](https://mlsanity.com/paper/1505.07818/) (2015)
- [Learning Transferable Features with Deep Adaptation Networks](https://mlsanity.com/paper/1502.02791/) (2015)
- [Adversarial Discriminative Domain Adaptation](https://mlsanity.com/paper/1702.05464/) (2017)
- [Deep CORAL: Correlation Alignment for Deep Domain Adaptation](https://mlsanity.com/paper/1607.01719/) (2016)
- [Deeper, Broader and Artier Domain Generalization](https://mlsanity.com/paper/1710.03077/) (2017)
- [Learning to Generalize: Meta-Learning for Domain Generalization](https://mlsanity.com/paper/1710.03463/) (2017)
- [Maximum Classifier Discrepancy for Unsupervised Domain Adaptation](https://mlsanity.com/paper/1712.02560/) (2017)
- [Deep Domain Confusion: Maximizing for Domain Invariance](https://mlsanity.com/paper/1412.3474/) (2014)
- [Contrastive Adaptation Network for Unsupervised Domain Adaptation](https://mlsanity.com/paper/1901.00976/) (2019)
- [Bridging Theory and Algorithm for Domain Adaptation](https://mlsanity.com/paper/1904.05801/) (2019)
- [Domain Separation Networks](https://mlsanity.com/paper/1608.06019/) (2016)
- [Unsupervised Pixel-Level Domain Adaptation with Generative Adversarial Networks](https://mlsanity.com/paper/1612.05424/) (2016)
- [A DIRT-T Approach to Unsupervised Domain Adaptation](https://mlsanity.com/paper/1802.08735/) (2018)
- [Progressive Feature Alignment for Unsupervised Domain Adaptation](https://mlsanity.com/paper/1811.08585/) (2018)
- [Domain Adaptation with Conditional Distribution Matching and Generalized Label Shift](https://mlsanity.com/paper/2003.04475/) (2020)
- [Unsupervised Domain Adaptation through Self-Supervision](https://mlsanity.com/paper/1909.11825/) (2019)
- [An introduction to domain adaptation and transfer learning](https://mlsanity.com/paper/1812.11806/) (2018)
- [A survey on domain adaptation theory: learning bounds and theoretical guarantees](https://mlsanity.com/paper/2004.11829/) (2020)
- [Domain Agnostic Learning with Disentangled Representations](https://mlsanity.com/paper/1904.12347/) (2019)
- [Support and Invertibility in Domain-Invariant Representations](https://mlsanity.com/paper/1903.03448/) (2019)
- [Meta-Learning Requires Meta-Augmentation](https://mlsanity.com/paper/2007.05549/) (2020)
- [MetaAlign: Coordinating Domain Alignment and Classification for Unsupervised Domain Adaptation](https://mlsanity.com/paper/2103.13575/) (2021)
- [d-SNE: Domain Adaptation using Stochastic Neighborhood Embedding](https://mlsanity.com/paper/1905.12775/) (2019)
- [Co-regularized Alignment for Unsupervised Domain Adaptation](https://mlsanity.com/paper/1811.05443/) (2018)
- [On Learning Invariant Representation for Domain Adaptation](https://mlsanity.com/paper/1901.09453/) (2019)
- [Unsupervised Domain Adaptation via Regularized Conditional Alignment](https://mlsanity.com/paper/1905.10885/) (2019)
- [Associative Domain Adaptation](https://mlsanity.com/paper/1708.00938/) (2017)
- [A New PAC-Bayesian Perspective on Domain Adaptation](https://mlsanity.com/paper/1506.04573/) (2015)
- [Robust Domain Adaptation: Representations, Weights and Inductive Bias](https://mlsanity.com/paper/2006.13629/) (2020)

## Built on by (most cited first, 1 of 1)

- [Jacobian Norm for Unsupervised Source-Free Domain Adaptation](https://mlsanity.com/paper/2204.03467/) (2022)

## Abstract (arXiv)

Domain alignment is currently the most prevalent solution to unsupervised domain-adaptation tasks and are often being presented as minimizers of some theoretical upper-bounds on risk in the target domain. However, further works revealed severe inadequacies between theory and practice: we consolidate this analysis and confirm that imposing domain invariance on features is neither necessary nor sufficient to obtain low target risk. We instead argue that successful deep domain adaptation rely largely on hidden inductive biases found in the common practice, such as model pre-training or design of encoder architecture. We perform various ablation experiments on popular benchmarks and our own synthetic transfers to illustrate their role in prototypical situations. To conclude our analysis, we propose to meta-learn parametric inductive biases to solve specific transfers and show their superior performance over handcrafted heuristics.