# Temporal Domain Generalization with Drift-Aware Dynamic Neural Networks

arXiv: https://arxiv.org/abs/2205.10664 · HTML: https://mlsanity.com/paper/2205.10664/

## Key facts

- Cited by 33 papers in the map (data through 14 Oct 2026).
- Builds on 27 of its 47 references that are in the map.
- First posted on arXiv 21 May 2022.

## Builds on (most cited first, 27 of 27)

- [An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale](https://mlsanity.com/paper/2010.11929/) (2020)
- [Microsoft COCO: Common Objects in Context](https://mlsanity.com/paper/1405.0312/) (2014)
- [Model-Agnostic Meta-Learning for Fast Adaptation of Deep Networks](https://mlsanity.com/paper/1703.03400/) (2017)
- [Domain-Adversarial Training of Neural Networks](https://mlsanity.com/paper/1505.07818/) (2015)
- [Invariant Risk Minimization](https://mlsanity.com/paper/1907.02893/) (2019)
- [Domain Randomization for Transferring Deep Neural Networks from Simulation to the Real World](https://mlsanity.com/paper/1703.06907/) (2017)
- [Adversarial Discriminative Domain Adaptation](https://mlsanity.com/paper/1702.05464/) (2017)
- [Deeper, Broader and Artier Domain Generalization](https://mlsanity.com/paper/1710.03077/) (2017)
- [Generalizing to Unseen Domains: A Survey on Domain Generalization](https://mlsanity.com/paper/2103.03097/) (2021)
- [Deep Visual Domain Adaptation: A Survey](https://mlsanity.com/paper/1802.03601/) (2018)
- [Domain Generalization via Invariant Feature Representation](https://mlsanity.com/paper/1301.2115/) (2013)
- [Self-Challenging Improves Cross-Domain Generalization](https://mlsanity.com/paper/2007.02454/) (2020)
- [Domain Generalization via Model-Agnostic Learning of Semantic Features](https://mlsanity.com/paper/1910.13580/) (2019)
- [Unified Deep Supervised Domain Adaptation and Generalization](https://mlsanity.com/paper/1709.10190/) (2017)
- [Learning to Learn Single Domain Generalization](https://mlsanity.com/paper/2003.13216/) (2020)
- [Training Deep Networks with Synthetic Data: Bridging the Reality Gap by Domain Randomization](https://mlsanity.com/paper/1804.06516/) (2018)
- [Generalizing Across Domains via Cross-Gradient Training](https://mlsanity.com/paper/1804.10745/) (2018)
- [Deep Visual Domain Adaptation](https://mlsanity.com/paper/2012.14176/) (2020)
- [Best sources forward: domain generalization through source-specific nets](https://mlsanity.com/paper/1806.05810/) (2018)
- [Continuously Indexed Domain Adaptation](https://mlsanity.com/paper/2007.01807/) (2020)
- [Graph Structure of Neural Networks](https://mlsanity.com/paper/2007.06559/) (2020)
- [A Unified Feature Disentangler for Multi-Domain Image Translation and Manipulation](https://mlsanity.com/paper/1809.01361/) (2018)
- [A law of robustness for two-layers neural networks](https://mlsanity.com/paper/2009.14444/) (2020)
- [Recent advances in deep learning theory](https://mlsanity.com/paper/2012.10931/) (2020)
- [AdaGraph: Unifying Predictive and Continuous Domain Adaptation through Graphs](https://mlsanity.com/paper/1903.07062/) (2019)
- [Training for the Future: A Simple Gradient Interpolation Loss to Generalize Along Time](https://mlsanity.com/paper/2108.06721/) (2021)
- [Sequential Learning for Domain Generalization](https://mlsanity.com/paper/2004.01377/) (2020)

## Built on by (most cited first, 30 of 33)

- [A Comprehensive Survey of Dynamic Graph Neural Networks: Models, Frameworks, Benchmarks, Experiments and Challenges](https://mlsanity.com/paper/2405.00476/) (2024)
- [Drift-Resilient TabPFN: In-Context Learning Temporal Distribution Shifts on Tabular Data](https://mlsanity.com/paper/2411.10634/) (2024)
- [Continuous Temporal Domain Generalization](https://mlsanity.com/paper/2405.16075/) (2024)
- [Learning Robust Spectral Dynamics for Temporal Domain Generalization](https://mlsanity.com/paper/2505.12585/) (2025)
- [DEJA VU: Continual Model Generalization For Unseen Domains](https://mlsanity.com/paper/2301.10418/) (2023)
- [Online Test-Time Adaptation of Spatial-Temporal Traffic Flow Forecasting](https://mlsanity.com/paper/2401.04148/) (2024)
- [Towards Counterfactual Fairness-aware Domain Generalization in Changing Environments](https://mlsanity.com/paper/2309.13005/) (2023)
- [Causal Graph ODE: Continuous Treatment Effect Modeling in Multi-agent Dynamical Systems](https://mlsanity.com/paper/2403.00178/) (2024)
- [Temporal Test-Time Adaptation with State-Space Models](https://mlsanity.com/paper/2407.12492/) (2024)
- [Graph Neural Network for spatiotemporal data: methods and applications](https://mlsanity.com/paper/2306.00012/) (2023)
- [CODA: Temporal Domain Generalization via Concept Drift Simulator](https://mlsanity.com/paper/2310.01508/) (2023)
- [POND: Multi-Source Time Series Domain Adaptation with Information-Aware Prompt Tuning](https://mlsanity.com/paper/2312.12276/) (2023)
- [Phase-driven Domain Generalizable Learning for Nonstationary Time Series](https://mlsanity.com/paper/2402.05960/) (2024)
- [Learning Time-Aware Causal Representation for Model Generalization in Evolving Domains](https://mlsanity.com/paper/2506.17718/) (2025)
- [Non-stationary Domain Generalization: Theory and Algorithm](https://mlsanity.com/paper/2405.06816/) (2024)
- [Rethinking Generalizable Infrared Small Target Detection: A Real-scene Benchmark and Cross-view Representation Learning](https://mlsanity.com/paper/2504.16487/) (2025)
- [Continuous Domain Generalization](https://mlsanity.com/paper/2505.13519/) (2025)
- [RCCDA: Adaptive Model Updates in the Presence of Concept Drift under a Constrained Resource Budget](https://mlsanity.com/paper/2505.24149/) (2025)
- [Optimal Empirical Risk Minimization under Temporal Distribution Shifts](https://mlsanity.com/paper/2507.13287/) (2025)
- [TEFL: Prediction-Residual-Guided Rolling Forecasting for Multi-Horizon Time Series](https://mlsanity.com/paper/2602.22520/) (2026)
- [Domain Generalization Deep Graph Transformation](https://mlsanity.com/paper/2305.11389/) (2023)
- [Temporal Representation Learning for Stock Similarities and Its Applications in Investment Management](https://mlsanity.com/paper/2407.13751/) (2024)
- [Wormhole: Concept-Aware Deep Representation Learning for Co-Evolving Sequences](https://mlsanity.com/paper/2409.13857/) (2024)
- [Koopman Operator Identification of Model Parameter Trajectories for Temporal Domain Generalization (KOMET)](https://mlsanity.com/paper/2603.26923/) (2026)
- [Prompting-based Temporal Domain Generalization](https://mlsanity.com/paper/2310.02473/) (2023)
- [ODEStream: A Buffer-Free Online Learning Framework with ODE-based Adaptor for Streaming Time Series Forecasting](https://mlsanity.com/paper/2411.07413/) (2024)
- [Learning Latent Spaces for Domain Generalization in Time Series Forecasting](https://mlsanity.com/paper/2412.11171/) (2024)
- [Temporal Generalization: A Reality Check](https://mlsanity.com/paper/2509.23487/) (2025)
- [Parameter-Efficient Temporal Domain Generalization](https://mlsanity.com/paper/2602.11965/) (2026)
- [When Can You Correct Distribution Drift in Temporal Graph Generation? A Sharpening--Drift Tension and an Impossibility for Observation-Based Correction](https://mlsanity.com/paper/2607.24662/) (2026)

## Abstract (arXiv)

Temporal domain generalization is a promising yet extremely challenging area where the goal is to learn models under temporally changing data distributions and generalize to unseen data distributions following the trends of the change. The advancement of this area is challenged by: 1) characterizing data distribution drift and its impacts on models, 2) expressiveness in tracking the model dynamics, and 3) theoretical guarantee on the performance. To address them, we propose a Temporal Domain Generalization with Drift-Aware Dynamic Neural Network (DRAIN) framework. Specifically, we formulate the problem into a Bayesian framework that jointly models the relation between data and model dynamics. We then build a recurrent graph generation scenario to characterize the dynamic graph-structured neural networks learned across different time points. It captures the temporal drift of model parameters and data distributions and can predict models in the future without the presence of future data. In addition, we explore theoretical guarantees of the model performance under the challenging temporal DG setting and provide theoretical analysis, including uncertainty and generalization error. Finally, extensive experiments on several real-world benchmarks with temporal drift demonstrate the effectiveness and efficiency of the proposed method.