# On the benefits of representation regularization in invariance based domain generalization arXiv: https://arxiv.org/abs/2105.14529 ยท HTML: https://mlsanity.com/paper/2105.14529/ ## Key facts - Cited by 17 papers in the map (data through 14 Oct 2026). - Builds on 28 of its 41 references that are in the map. - First posted on arXiv 30 May 2021. ## Builds on (most cited first, 28 of 28) - [Explaining and Harnessing Adversarial Examples](https://mlsanity.com/paper/1412.6572/) (2014) - [mixup: Beyond Empirical Risk Minimization](https://mlsanity.com/paper/1710.09412/) (2017) - [Domain-Adversarial Training of Neural Networks](https://mlsanity.com/paper/1505.07818/) (2015) - [Conditional Generative Adversarial Nets](https://mlsanity.com/paper/1411.1784/) (2014) - [Invariant Risk Minimization](https://mlsanity.com/paper/1907.02893/) (2019) - [Spectral Normalization for Generative Adversarial Networks](https://mlsanity.com/paper/1802.05957/) (2018) - [Deep Hashing Network for Unsupervised Domain Adaptation](https://mlsanity.com/paper/1706.07522/) (2017) - [In Search of Lost Domain Generalization](https://mlsanity.com/paper/2007.01434/) (2020) - [Deeper, Broader and Artier Domain Generalization](https://mlsanity.com/paper/1710.03077/) (2017) - [Learning to Generalize: Meta-Learning for Domain Generalization](https://mlsanity.com/paper/1710.03463/) (2017) - [Domain Generalization with MixStyle](https://mlsanity.com/paper/2104.02008/) (2021) - [Generalizing to Unseen Domains via Adversarial Data Augmentation](https://mlsanity.com/paper/1805.12018/) (2018) - [Learning to Generate Novel Domains for Domain Generalization](https://mlsanity.com/paper/2007.03304/) (2020) - [Emergence of Invariance and Disentanglement in Deep Representations](https://mlsanity.com/paper/1706.01350/) (2017) - [Domain Generalization Using a Mixture of Multiple Latent Domains](https://mlsanity.com/paper/1911.07661/) (2019) - [Domain Generalization via Conditional Invariant Representation](https://mlsanity.com/paper/1807.08479/) (2018) - [Feature-Critic Networks for Heterogeneous Domain Generalization](https://mlsanity.com/paper/1901.11448/) (2019) - [Learning Deep Kernels for Non-Parametric Two-Sample Tests](https://mlsanity.com/paper/2002.09116/) (2020) - [Invariance, Causality and Robustness](https://mlsanity.com/paper/1812.08233/) (2018) - [Does Invariant Risk Minimization Capture Invariance?](https://mlsanity.com/paper/2101.01134/) (2021) - [DIVA: Domain Invariant Variational Autoencoders](https://mlsanity.com/paper/1905.10427/) (2019) - [Generalizing to unseen domains via distribution matching](https://mlsanity.com/paper/1911.00804/) (2019) - [Domain Adversarial Neural Networks for Domain Generalization: When It Works and How to Improve](https://mlsanity.com/paper/2102.03924/) (2021) - [Nonlinear Invariant Risk Minimization: A Causal Approach](https://mlsanity.com/paper/2102.12353/) (2021) - [A Generalization Error Bound for Multi-class Domain Generalization](https://mlsanity.com/paper/1905.10392/) (2019) - [Learning Robust Models Using The Principle of Independent Causal Mechanisms](https://mlsanity.com/paper/2010.07167/) (2020) - [DomainMix: Learning Generalizable Person Re-Identification Without Human Annotations](https://mlsanity.com/paper/2011.11953/) (2020) - [SGD Implicitly Regularizes Generalization Error](https://mlsanity.com/paper/2104.04874/) (2021) ## Built on by (most cited first, 17 of 17) - [Fairness and Accuracy under Domain Generalization](https://mlsanity.com/paper/2301.13323/) (2023) - [Fair Representation Learning through Implicit Path Alignment](https://mlsanity.com/paper/2205.13316/) (2022) - [Style-Hallucinated Dual Consistency Learning: A Unified Framework for Visual Domain Generalization](https://mlsanity.com/paper/2212.09068/) (2022) - [Variational Disentanglement for Domain Generalization](https://mlsanity.com/paper/2109.05826/) (2021) - [Generalizing across Temporal Domains with Koopman Operators](https://mlsanity.com/paper/2402.07834/) (2024) - [Foresee What You Will Learn: Data Augmentation for Domain Generalization in Non-stationary Environment](https://mlsanity.com/paper/2301.07845/) (2023) - [Unknown Domain Inconsistency Minimization for Domain Generalization](https://mlsanity.com/paper/2403.07329/) (2024) - [Learning Optimal Features via Partial Invariance](https://mlsanity.com/paper/2301.12067/) (2023) - [Gap Minimization for Knowledge Sharing and Transfer](https://mlsanity.com/paper/2201.11231/) (2022) - [Regularization Penalty Optimization for Addressing Data Quality Variance in OoD Algorithms](https://mlsanity.com/paper/2206.05749/) (2022) - [Diversity-enhancing Generative Network for Few-shot Hypothesis Adaptation](https://mlsanity.com/paper/2307.05948/) (2023) - [Evolving Domain Generalization](https://mlsanity.com/paper/2206.00047/) (2022) - [Repeated Environment Inference for Invariant Learning](https://mlsanity.com/paper/2207.12876/) (2022) - [Feature-aligned N-BEATS with Sinkhorn divergence](https://mlsanity.com/paper/2305.15196/) (2023) - [Guidance Not Obstruction: A Conjugate Consistent Enhanced Strategy for Domain Generalization](https://mlsanity.com/paper/2412.10089/) (2024) - [Connecting Domains and Contrasting Samples: A Ladder for Domain Generalization](https://mlsanity.com/paper/2510.16704/) (2025) - [Why Invariance is Not Enough for Biomedical Domain Generalization and How to Fix It](https://mlsanity.com/paper/2604.02564/) (2026) ## Abstract (arXiv) A crucial aspect in reliable machine learning is to design a deployable system in generalizing new related but unobserved environments. Domain generalization aims to alleviate such a prediction gap between the observed and unseen environments. Previous approaches commonly incorporated learning invariant representation for achieving good empirical performance. In this paper, we reveal that merely learning invariant representation is vulnerable to the unseen environment. To this end, we derive novel theoretical analysis to control the unseen test environment error in the representation learning, which highlights the importance of controlling the smoothness of representation. In practice, our analysis further inspires an efficient regularization method to improve the robustness in domain generalization. Our regularization is orthogonal to and can be straightforwardly adopted in existing domain generalization algorithms for invariant representation learning. Empirical results show that our algorithm outperforms the base versions in various dataset and invariance criteria.