# Non-stationary Domain Generalization: Theory and Algorithm

arXiv: https://arxiv.org/abs/2405.06816 · HTML: https://mlsanity.com/paper/2405.06816/

## Key facts

- Cited by 4 papers in the map (data through 14 Oct 2026).
- Builds on 39 of its 54 references that are in the map.
- First posted on arXiv 10 May 2024.

## Builds on (most cited first, 30 of 39)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [mixup: Beyond Empirical Risk Minimization](https://mlsanity.com/paper/1710.09412/) (2017)
- [Overcoming catastrophic forgetting in neural networks](https://mlsanity.com/paper/1612.00796/) (2016)
- [Domain-Adversarial Training of Neural Networks](https://mlsanity.com/paper/1505.07818/) (2015)
- [Invariant Risk Minimization](https://mlsanity.com/paper/1907.02893/) (2019)
- [WILDS: A Benchmark of in-the-Wild Distribution Shifts](https://mlsanity.com/paper/2012.07421/) (2020)
- [Deep CORAL: Correlation Alignment for Deep Domain Adaptation](https://mlsanity.com/paper/1607.01719/) (2016)
- [Efficient Lifelong Learning with A-GEM](https://mlsanity.com/paper/1812.00420/) (2018)
- [PackNet: Adding Multiple Tasks to a Single Network by Iterative Pruning](https://mlsanity.com/paper/1711.05769/) (2017)
- [Learning to Generalize: Meta-Learning for Domain Generalization](https://mlsanity.com/paper/1710.03463/) (2017)
- [Functional Map of the World](https://mlsanity.com/paper/1711.07846/) (2017)
- [Learning to Learn Single Domain Generalization](https://mlsanity.com/paper/2003.13216/) (2020)
- [Learning to Generate Novel Domains for Domain Generalization](https://mlsanity.com/paper/2007.03304/) (2020)
- [Gradient Matching for Domain Generalization](https://mlsanity.com/paper/2104.09937/) (2021)
- [SelfReg: Self-supervised Contrastive Regularization for Domain Generalization](https://mlsanity.com/paper/2104.09841/) (2021)
- [Fishr: Invariant Gradient Variances for Out-of-Distribution Generalization](https://mlsanity.com/paper/2109.02934/) (2021)
- [Understanding Self-Training for Gradual Domain Adaptation](https://mlsanity.com/paper/2002.11361/) (2020)
- [Disentangled Sequential Autoencoder](https://mlsanity.com/paper/1803.02991/) (2018)
- [Generalizing to unseen domains via distribution matching](https://mlsanity.com/paper/1911.00804/) (2019)
- [The CLEAR Benchmark: Continual LEArning on Real-World Imagery](https://mlsanity.com/paper/2201.06289/) (2022)
- [Continuously Indexed Domain Adaptation](https://mlsanity.com/paper/2007.01807/) (2020)
- [Self-training Avoids Using Spurious Features Under Domain Shift](https://mlsanity.com/paper/2006.10032/) (2020)
- [A Century of Portraits: A Visual Historical Record of American High School Yearbooks](https://mlsanity.com/paper/1511.02575/) (2015)
- [Domain Invariant Representation Learning with Domain Density Transformations](https://mlsanity.com/paper/2102.05082/) (2021)
- [Feature Stylization and Domain-aware Contrastive Learning for Domain Generalization](https://mlsanity.com/paper/2108.08596/) (2021)
- [KL Guided Domain Adaptation](https://mlsanity.com/paper/2106.07780/) (2021)
- [Temporal Domain Generalization with Drift-Aware Dynamic Neural Networks](https://mlsanity.com/paper/2205.10664/) (2022)
- [Some Theoretical Properties of GANs](https://mlsanity.com/paper/1803.07819/) (2018)
- [Gradual Domain Adaptation without Indexed Intermediate Domains](https://mlsanity.com/paper/2207.04587/) (2022)

## Built on by (most cited first, 4 of 4)

- [Optimal Empirical Risk Minimization under Temporal Distribution Shifts](https://mlsanity.com/paper/2507.13287/) (2025)
- [FedDAA: Dynamic Client Clustering for Concept Drift Adaptation in Federated Learning](https://mlsanity.com/paper/2506.21054/) (2025)
- [Open-Set Heterogeneous Domain Adaptation: Theoretical Analysis and Algorithm](https://mlsanity.com/paper/2412.13036/) (2024)
- [From Relevance to Validity: Zero-Shot Tool Selection via Functional and Contextual Validation](https://mlsanity.com/paper/2610.20388/) (2026)

## Abstract (arXiv)

Although recent advances in machine learning have shown its success to learn from independent and identically distributed (IID) data, it is vulnerable to out-of-distribution (OOD) data in an open world. Domain generalization (DG) deals with such an issue and it aims to learn a model from multiple source domains that can be generalized to unseen target domains. Existing studies on DG have largely focused on stationary settings with homogeneous source domains. However, in many applications, domains may evolve along a specific direction (e.g., time, space). Without accounting for such non-stationary patterns, models trained with existing methods may fail to generalize on OOD data. In this paper, we study domain generalization in non-stationary environment. We first examine the impact of environmental non-stationarity on model performance and establish the theoretical upper bounds for the model error at target domains. Then, we propose a novel algorithm based on adaptive invariant representation learning, which leverages the non-stationary pattern to train a model that attains good performance on target domains. Experiments on both synthetic and real data validate the proposed algorithm.