mlsanityData to 14 Oct 2026

Learning Latent Spaces for Domain Generalization in Time Series Forecasting

arXiv 2412.11171 ↗First posted 15 Dec 2024cs.LGcited by 0

Not on the map. Why?

Abstract (arXiv)

Time series forecasting is vital in many real-world applications, yet developing models that generalize well on unseen relevant domains -- such as forecasting web traffic data on new platforms/websites or estimating e-commerce demand in new regions -- remains underexplored. Existing forecasting models often struggle with domain shifts in time series data, as the temporal patterns involve complex components like trends, seasonality, etc. While some prior work addresses this by matching feature distributions across domains or disentangling domain-shared features using label information, they fail to reveal insights into the latent temporal dependencies, which are critical for identifying common patterns across domains and achieving generalization. We propose a framework for domain generalization in time series forecasting by mining the latent factors that govern temporal dependencies across domains. Our approach uses a decomposition-based architecture with a new Conditional β-Variational Autoencoder (VAE), wherein time series data is first decomposed into trend-cyclical and seasonal components, each modeled independently through separate β-VAE modules. The β-VAE aims to capture disentangled latent factors that control temporal dependencies across domains. We enhance the learning of domain-specific information with a decoder-conditional design and introduce domain regularization to improve the separation of domain-shared and domain-specific latent factors. Our proposed method is flexible and can be applied to various time series forecasting models, enabling effective domain generalization with simplicity and efficiency. We validate its effectiveness on five real-world time series datasets, covering web traffic, e-commerce, finance and power consumption, demonstrating improved generalization performance over state-of-the-art methods.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Builds on 40 of its 59 references that are in the map. First posted on arXiv 15 Dec 2024.

Builds on

40 of 59 references.

References · 40

  1. 2017Attention Is All You Needcited by 60,361
  2. 2019PyTorch: An Imperative Style, High-Performance Deep Learning Librarycited by 20,305
  3. 2014Neural Machine Translation by Jointly Learning to Align and Translatecited by 4,548
  4. 2015Domain-Adversarial Training of Neural Networkscited by 3,044
  5. 2023A Survey of Large Language Modelscited by 2,576
  6. 2014Empirical Evaluation of Gated Recurrent Neural Networks on Sequence Modelingcited by 2,367
  7. 2020Informer: Beyond Efficient Transformer for Long Sequence Time-Series Forecastingcited by 2,169
  8. 2022A Time Series is Worth 64 Words: Long-term Forecasting with Transformerscited by 2,039
  9. 2016WaveNet: A Generative Model for Raw Audiocited by 1,722
  10. 2021Autoformer: Decomposition Transformers with Auto-Correlation for Long-Term Series Forecastingcited by 1,676
  11. 2018An Empirical Evaluation of Generic Convolutional and Recurrent Networks for Sequence Modelingcited by 1,476
  12. 2022Are Transformers Effective for Time Series Forecasting?cited by 1,424
28 more
  1. 2022FEDformer: Frequency Enhanced Decomposed Transformer for Long-term Series Forecastingcited by 1,210
  2. 2020In Search of Lost Domain Generalizationcited by 1,054
  3. 2017DeepAR: Probabilistic Forecasting with Autoregressive Recurrent Networkscited by 1,013
  4. 2021Generalizing to Unseen Domains: A Survey on Domain Generalizationcited by 766
  5. 2024Chronos: Learning the Language of Time Seriescited by 759
  6. 2023Time-LLM: Time Series Forecasting by Reprogramming Large Language Modelscited by 700
  7. 2023One Fits All:Power General Time Series Analysis by Pretrained LMcited by 632
  8. 2024Unified Training of Universal Time Series Forecasting Transformerscited by 614
  9. 2019Enhancing the Locality and Breaking the Memory Bottleneck of Transformer on Time Series Forecastingcited by 607
  10. 2020Time Series Forecasting With Deep Learning: A Surveycited by 459
  11. 2021Towards Out-Of-Distribution Generalization: A Surveycited by 436
  12. 2021Gradient Matching for Domain Generalizationcited by 258
  13. 2019Domain Randomization and Pyramid Consistency: Simulation-to-Real Generalization without Accessing Target Domain Datacited by 246
  14. 2018DLOW: Domain Flow for Adaptation and Generalizationcited by 177
  15. 2024Are Language Models Actually Useful for Time Series Forecasting?cited by 168
  16. 2023UniTime: A Language-Empowered Unified Model for Cross-Domain Time Series Forecastingcited by 144
  17. 2021AdaRNN: Adaptive Learning and Forecasting of Time Seriescited by 121
  18. 2019DIVA: Domain Invariant Variational Autoencoderscited by 115
  19. 2022State-of-the-art generalisation research in NLP: A taxonomy and reviewcited by 98
  20. 2023Dish-TS: A General Paradigm for Alleviating Distribution Shift in Time Series Forecastingcited by 93
  21. 2022Temporal Domain Generalization with Drift-Aware Dynamic Neural Networkscited by 33
  22. 2022Out-of-Distribution Representation Learning for Time Series Classificationcited by 31
  23. 2021Training for the Future: A Simple Gradient Interpolation Loss to Generalize Along Timecited by 28
  24. 2024Revisiting VAE for Unsupervised Time Series Anomaly Detection: A Frequency Perspectivecited by 24
  25. 2021Causal-based Time Series Domain Generalization for Vehicle Intention Predictioncited by 17
  26. 2023Revisiting Structured Variational Autoencoderscited by 11
  27. 2021Parameter Efficient Deep Probabilistic Forecastingcited by 8
  28. 2022Domain Generalization via Selective Consistency Regularization for Time Series Classificationcited by 1

Built on by

Not cited yet.