mlsanityData to 7 Oct 2026

Learning Pattern-Specific Experts for Time Series Forecasting Under Patch-level Distribution Shift

arXiv 2410.09836 ↗First posted 13 Oct 2024cs.LG stat.MLcited by 23

Not on the map. Why?

Abstract (arXiv)

Time series forecasting, which aims to predict future values based on historical data, has garnered significant attention due to its broad range of applications. However, real-world time series often exhibit complex non-uniform distribution with varying patterns across segments, such as season, operating condition, or semantic meaning, making accurate forecasting challenging. Existing approaches, which typically train a single model to capture all these diverse patterns, often struggle with the pattern drifts between patches and may lead to poor generalization. To address these challenges, we propose TFPS, a novel architecture that leverages pattern-specific experts for more accurate and adaptable time series forecasting. TFPS employs a dual-domain encoder to capture both time-domain and frequency-domain features, enabling a more comprehensive understanding of temporal dynamics. It then uses subspace clustering to dynamically identify distinct patterns across data patches. Finally, pattern-specific experts model these unique patterns, delivering tailored predictions for each patch. By explicitly learning and adapting to evolving patterns, TFPS achieves significantly improved forecasting accuracy. Extensive experiments on real-world datasets demonstrate that TFPS outperforms state-of-the-art methods, particularly in long-term forecasting, through its dynamic and pattern-aware learning approach. The data and codes are available: https://github.com/syrGitHub/TFPS.

Key facts. Cited by 23 papers in the map (data through 7 Oct 2026). Builds on 45 of its 75 references that are in the map. First posted on arXiv 13 Oct 2024.

Builds on

45 of 75 references.

References · 45

  1. 2017Attention Is All You Needcited by 59,932
  2. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,204
  3. 2019PyTorch: An Imperative Style, High-Performance Deep Learning Librarycited by 20,160
  4. 2017Outrageously Large Neural Networks: The Sparsely-Gated Mixture-of-Experts Layercited by 3,413
  5. 2021Switch Transformers: Scaling to Trillion Parameter Models with Simple and Efficient Sparsitycited by 3,014
  6. 2020Informer: Beyond Efficient Transformer for Long Sequence Time-Series Forecastingcited by 2,141
  7. 2022A Time Series is Worth 64 Words: Long-term Forecasting with Transformerscited by 1,981
  8. 2021Autoformer: Decomposition Transformers with Auto-Correlation for Long-Term Series Forecastingcited by 1,654
  9. 2022Are Transformers Effective for Time Series Forecasting?cited by 1,400
  10. 2022FEDformer: Frequency Enhanced Decomposed Transformer for Long-term Series Forecastingcited by 1,200
  11. 2022TimesNet: Temporal 2D-Variation Modeling for General Time Series Analysiscited by 1,030
  12. 2023iTransformer: Inverted Transformers Are Effective for Time Series Forecastingcited by 906
33 more
  1. 2021GLaM: Efficient Scaling of Language Models with Mixture-of-Expertscited by 890
  2. 2017Modeling Long- and Short-Term Temporal Patterns with Deep Neural Networkscited by 869
  3. 2024DeepSeekMoE: Towards Ultimate Expert Specialization in Mixture-of-Experts Language Modelscited by 834
  4. 2023Time-LLM: Time Series Forecasting by Reprogramming Large Language Modelscited by 680
  5. 2024Unified Training of Universal Time Series Forecasting Transformerscited by 588
  6. 2020Learning under Concept Drift: A Reviewcited by 470
  7. 2024MOMENT: A Family of Open Time-series Foundation Modelscited by 453
  8. 2022Non-stationary Transformers: Exploring the Stationarity in Time Series Forecastingcited by 385
  9. 2021FNet: Mixing Tokens with Fourier Transformscited by 343
  10. 2024MACE: Mass Concept Erasure in Diffusion Modelscited by 298
  11. 2023From Sparse to Soft Mixtures of Expertscited by 245
  12. 2024TFB: Towards Comprehensive and Fair Benchmarking of Time Series Forecasting Methodscited by 242
  13. 2024Timer: Generative Pre-trained Transformers Are Large Time Series Modelscited by 198
  14. 2023FITS: Modeling Time Series with 10k Parameterscited by 185
  15. 2023TF-ICON: Diffusion-Based Training-Free Cross-Domain Image Compositioncited by 182
  16. 2015On Wasserstein Two Sample Testing and Related Families of Nonparametric Testscited by 165
  17. 2023TSMixer: Lightweight MLP-Mixer Model for Multivariate Time Series Forecastingcited by 136
  18. 2023Exploring Progress in Multivariate Time Series Forecasting: Comprehensive Benchmarking and Heterogeneity Analysiscited by 113
  19. 2023Dish-TS: A General Paradigm for Alleviating Distribution Shift in Time Series Forecastingcited by 91
  20. 2023OneNet: Enhancing Time Series Forecasting Models under Concept Drift by Online Ensemblingcited by 71
  21. 2024TSLANet: Rethinking Transformers for Time Series Representation Learningcited by 71
  22. 2024MoE-Mamba: Efficient Selective State Space Models with Mixture of Expertscited by 60
  23. 2021Person Re-identification Method Based on Color Attack and Joint Defencecited by 56
  24. 2024Cross-Modality Perturbation Synergy Attack for Person Re-identificationcited by 45
  25. 2021Eliminate Deviation with Deviation for Data Augmentation and a General Multi-modal Data Learning Methodcited by 42
  26. 2019Divide and Conquer the Embedding Space for Metric Learningcited by 39
  27. 2024BlackMamba: Mixture of Experts for State-Space Modelscited by 37
  28. 2023Mixture-of-Linear-Experts for Long-term Time Series Forecastingcited by 29
  29. 2023Cross-city Few-Shot Traffic Forecasting via Traffic Pattern Bankcited by 22
  30. 2023An Accurate and Interpretable Framework for Trustworthy Process Monitoringcited by 18
  31. 2023TFDNet: Time-Frequency Enhanced Decomposed Network for Long-term Time Series Forecastingcited by 7
  32. 2024Continual Traffic Forecasting via Mixture of Expertscited by 6
  33. 2023A Joint Time-frequency Domain Transformer for Multivariate Time Series Forecastingcited by 5

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2025: 9; 2026: 14.

23 papers.

Sort:

By area:

  1. 2025Patch-wise Structural Loss for Time Series Forecastingcited by 34
  2. 2025Out-of-Distribution Generalization in Time Series: A Surveycited by 15
  3. 2025ASTGI: Adaptive Spatio-Temporal Graph Interactions for Irregular Multivariate Time Series Forecastingcited by 7
  4. 2025Diffusion-Based Image Editing: An Unforeseen Adversary to Robust Invisible Watermarkscited by 2
  5. 2026Vanishing Watermarks: Diffusion-Based Image Editing Undermines Robust Invisible Watermarkingcited by 2
  6. 2025Time Tracker: Mixture-of-Experts-Enhanced Foundation Time Series Forecasting Model with Decoupled Training Pipelinescited by 1
  7. 2025Beyond All-to-All: Causal-Aligned Transformer with Dynamic Structure Learning for Multivariate Time Series Forecastingcited by 1
  8. 2025N-BEATS-MOE: N-BEATS with a Mixture-of-Experts Layer for Heterogeneous Time Series Forecastingcited by 1
  9. 2025SEED: Spectral Entropy-Guided Evaluation of SpatialTemporal Dependencies for Multivariate Time Series Forecastingcited by 1
  10. 2026Multi-Modal Time Series Prediction via Mixture of Modulated Expertscited by 1
  11. 2026Editing on the Generative Manifold: A Theoretical and Empirical Study of General Diffusion-Based Image Editing Trade-offscited by 1
  12. 2025Adapting LLMs to Time Series Forecasting via Temporal Heterogeneity Modeling and Representation Alignmentcited by 0
11 more
  1. 2026Manifold-Aware Temporal Domain Generalization for Large Language Modelscited by 0
  2. 2026When Denoising Becomes Unsigning: Theoretical and Empirical Analysis of Watermark Fragility Under Diffusion-Based Image Editingcited by 0
  3. 2026Editing Away the Evidence: Diffusion-Based Image Manipulation and the Failure Modes of Robust Watermarkingcited by 0
  4. 2026Dynamic TMoE: A Drift-Aware Dynamic Mixture of Experts Framework for Non-Stationary Time Series Forecastingcited by 0
  5. 2026On the Controllability-Fidelity Frontier in Diffusion Editingcited by 0
  6. 2026CausalMoE: A Billion-Scale Multimodal Foundation Model for Granger Causal Discovery with Pattern-Routed Heterogeneous Expertscited by 0
  7. 2026Towards a Unified Generative Model for Scarce Time Series with Domain Expertscited by 0
  8. 2026RAVEN: A Regime-Aware Variable-context Expert Network for Financial Time Series Forecastingcited by 0
  9. 2026Towards Reliable Zero-Shot Crowd Forecasting: Evaluating Time Series Foundation Models for Special Event Pedestrian Forecastingcited by 0
  10. 2026Which Histories Matter for Time Series Forecasting? Learning Predictive Relevance with Future Supervisioncited by 0
  11. 2026Aurora-X: Built for Extreme Time Series Forecastingcited by 0