# Time-Series Forecasting for Out-of-Distribution Generalization Using Invariant Learning

arXiv: https://arxiv.org/abs/2406.09130 · HTML: https://mlsanity.com/paper/2406.09130/

## Key facts

- Cited by 27 papers in the map (data through 14 Oct 2026).
- Builds on 32 of its 57 references that are in the map.
- First posted on arXiv 13 Jun 2024.

## Builds on (most cited first, 30 of 32)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [A Time Series is Worth 64 Words: Long-term Forecasting with Transformers](https://mlsanity.com/paper/2211.14730/) (2022)
- [Invariant Risk Minimization](https://mlsanity.com/paper/1907.02893/) (2019)
- [Autoformer: Decomposition Transformers with Auto-Correlation for Long-Term Series Forecasting](https://mlsanity.com/paper/2106.13008/) (2021)
- [Reformer: The Efficient Transformer](https://mlsanity.com/paper/2001.04451/) (2020)
- [Distributionally Robust Neural Networks for Group Shifts: On the Importance of Regularization for Worst-Case Generalization](https://mlsanity.com/paper/1911.08731/) (2019)
- [FEDformer: Frequency Enhanced Decomposed Transformer for Long-term Series Forecasting](https://mlsanity.com/paper/2201.12740/) (2022)
- [TimesNet: Temporal 2D-Variation Modeling for General Time Series Analysis](https://mlsanity.com/paper/2210.02186/) (2022)
- [Modeling Long- and Short-Term Temporal Patterns with Deep Neural Networks](https://mlsanity.com/paper/1703.07015/) (2017)
- [Out-of-Distribution Generalization via Risk Extrapolation (REx)](https://mlsanity.com/paper/2003.00688/) (2020)
- [Causal inference using invariant prediction: identification and confidence intervals](https://mlsanity.com/paper/1501.01332/) (2015)
- [Large Language Models Are Zero-Shot Time Series Forecasters](https://mlsanity.com/paper/2310.07820/) (2023)
- [Time Series Forecasting With Deep Learning: A Survey](https://mlsanity.com/paper/2004.13408/) (2020)
- [Towards Out-Of-Distribution Generalization: A Survey](https://mlsanity.com/paper/2108.13624/) (2021)
- [Non-stationary Transformers: Exploring the Stationarity in Time Series Forecasting](https://mlsanity.com/paper/2205.14415/) (2022)
- [Environment Inference for Invariant Learning](https://mlsanity.com/paper/2010.07249/) (2020)
- [Financial Time Series Forecasting with Deep Learning : A Systematic Literature Review: 2005-2019](https://mlsanity.com/paper/1911.13288/) (2019)
- [Gradient Starvation: A Learning Proclivity in Neural Networks](https://mlsanity.com/paper/2011.09468/) (2020)
- [Invariance Principle Meets Information Bottleneck for Out-of-Distribution Generalization](https://mlsanity.com/paper/2106.06607/) (2021)
- [Invariant Rationalization](https://mlsanity.com/paper/2003.09772/) (2020)
- [AdaRNN: Adaptive Learning and Forecasting of Time Series](https://mlsanity.com/paper/2108.04443/) (2021)
- [Dish-TS: A General Paradigm for Alleviating Distribution Shift in Time Series Forecasting](https://mlsanity.com/paper/2302.14829/) (2023)
- [Heterogeneous Risk Minimization](https://mlsanity.com/paper/2105.03818/) (2021)
- [OneNet: Enhancing Time Series Forecasting Models under Concept Drift by Online Ensembling](https://mlsanity.com/paper/2309.12659/) (2023)
- [ZIN: When and How to Learn Invariance Without Environment Partition?](https://mlsanity.com/paper/2203.05818/) (2022)
- [When is invariance useful in an Out-of-Distribution Generalization problem ?](https://mlsanity.com/paper/2008.01883/) (2020)
- [Out-of-Distribution Representation Learning for Time Series Classification](https://mlsanity.com/paper/2209.07027/) (2022)
- [DIVERSIFY: A General Framework for Time Series Out-of-distribution Detection and Generalization](https://mlsanity.com/paper/2308.02282/) (2023)
- [When in Doubt: Neural Non-Parametric Uncertainty Quantification for Epidemic Forecasting](https://mlsanity.com/paper/2106.03904/) (2021)
- [Performative Time-Series Forecasting](https://mlsanity.com/paper/2310.06077/) (2023)

## Built on by (most cited first, 27 of 27)

- [Out-of-Distribution Generalization in Time Series: A Survey](https://mlsanity.com/paper/2503.13868/) (2025)
- [Improving Time Series Forecasting via Instance-aware Post-hoc Revision](https://mlsanity.com/paper/2505.23583/) (2025)
- [Selective Learning for Deep Time Series Forecasting](https://mlsanity.com/paper/2510.25207/) (2025)
- [Evaluating the Generalization Ability of Spatiotemporal Model in Urban Scenario](https://mlsanity.com/paper/2410.04740/) (2024)
- [TimeRecipe: A Time-Series Forecasting Recipe via Benchmarking Module Level Effectiveness](https://mlsanity.com/paper/2506.06482/) (2025)
- [DELPHYNE: A Pre-Trained Model for General and Financial Time Series](https://mlsanity.com/paper/2506.06288/) (2025)
- [FredNormer: Frequency Domain Normalization for Non-stationary Time Series Forecasting](https://mlsanity.com/paper/2410.01860/) (2024)
- [EventTSF: Event-Aware Non-Stationary Time Series Forecasting](https://mlsanity.com/paper/2508.13434/) (2025)
- [Towards Generalizable PDE Dynamics Forecasting via Physics-Guided Invariant Learning](https://mlsanity.com/paper/2509.24332/) (2025)
- [TIFO: Time-Invariant Frequency Operator for Stationarity-Aware Representation Learning in Time Series](https://mlsanity.com/paper/2602.17122/) (2026)
- [Rethinking Post-Training Recipes for Multimodal Time-Series Forecasting](https://mlsanity.com/paper/2605.29401/) (2026)
- [Rating Quality of Diverse Time Series Data by Meta-learning from LLM Judgment](https://mlsanity.com/paper/2506.01290/) (2025)
- [TAT: Temporal-Aligned Transformer for Multi-Horizon Peak Demand Forecasting](https://mlsanity.com/paper/2507.10349/) (2025)
- [Inner-Instance Normalization for Time Series Forecasting](https://mlsanity.com/paper/2510.08657/) (2025)
- [Tackling Time-Series Forecasting Generalization via Mitigating Concept Drift](https://mlsanity.com/paper/2510.14814/) (2025)
- [CoRe: Coherency Regularization for Hierarchical Time Series](https://mlsanity.com/paper/2502.15983/) (2025)
- [CANet: ChronoAdaptive Network for Enhanced Long-Term Time Series Forecasting under Non-Stationarity](https://mlsanity.com/paper/2504.17913/) (2025)
- [Lightweight Time Series Data Valuation on Time Series Foundation Models via In-Context Finetuning](https://mlsanity.com/paper/2511.11648/) (2025)
- [TimeSAE: Causal Sparse Decoding for Faithful Explanations of Black-Box Time Series Models](https://mlsanity.com/paper/2601.09776/) (2026)
- [Cross-Machine Anomaly Detection Leveraging Pre-trained Time-series Model](https://mlsanity.com/paper/2604.05335/) (2026)
- [Autonomous Drift Learning in Data Streams: A Unified Perspective](https://mlsanity.com/paper/2605.01295/) (2026)
- [STEPS: A Temporal Smooth Error Propagation Solver on the Manifolds for Test-Time Adaptation in Time Series Forecasting](https://mlsanity.com/paper/2605.08005/) (2026)
- [SURGE: An Event-Centric Social Media Sentiment Time Series Benchmark with Interaction Structure](https://mlsanity.com/paper/2605.21198/) (2026)
- [VLBM: Variational Latent Basis Modeling for OOD Robust Multivariate Time Series Forecasting](https://mlsanity.com/paper/2606.02138/) (2026)
- [Conditionally Identifiable Latent-Environment Modeling for Out-of-Distribution Recommendation](https://mlsanity.com/paper/2608.03647/) (2026)
- [RAEGL: Risk-Aware Evidence-Gated Learning for Selective Contextual Routing under Temporal Shift](https://mlsanity.com/paper/2609.33340/) (2026)
- [TimeRSI: Towards Self-Improved Agentic Deployment for Time-Series Foundation Models](https://mlsanity.com/paper/2610.19725/) (2026)

## Abstract (arXiv)

Time-series forecasting (TSF) finds broad applications in real-world scenarios. Due to the dynamic nature of time-series data, it is crucial to equip TSF models with out-of-distribution (OOD) generalization abilities, as historical training data and future test data can have different distributions. In this paper, we aim to alleviate the inherent OOD problem in TSF via invariant learning. We identify fundamental challenges of invariant learning for TSF. First, the target variables in TSF may not be sufficiently determined by the input due to unobserved core variables in TSF, breaking the conventional assumption of invariant learning. Second, time-series datasets lack adequate environment labels, while existing environmental inference methods are not suitable for TSF. To address these challenges, we propose FOIL, a model-agnostic framework that enables timeseries Forecasting for Out-of-distribution generalization via Invariant Learning. FOIL employs a novel surrogate loss to mitigate the impact of unobserved variables. Further, FOIL implements a joint optimization by alternately inferring environments effectively with a multi-head network while preserving the temporal adjacency structure, and learning invariant representations across inferred environments for OOD generalized TSF. We demonstrate that the proposed FOIL significantly improves the performance of various TSF models, achieving gains of up to 85%.