Learning Pattern-Specific Experts for Time Series Forecasting Under Patch-level Distribution Shift
Not on the map. Why?
Abstract (arXiv)
Time series forecasting, which aims to predict future values based on historical data, has garnered significant attention due to its broad range of applications. However, real-world time series often exhibit complex non-uniform distribution with varying patterns across segments, such as season, operating condition, or semantic meaning, making accurate forecasting challenging. Existing approaches, which typically train a single model to capture all these diverse patterns, often struggle with the pattern drifts between patches and may lead to poor generalization. To address these challenges, we propose TFPS, a novel architecture that leverages pattern-specific experts for more accurate and adaptable time series forecasting. TFPS employs a dual-domain encoder to capture both time-domain and frequency-domain features, enabling a more comprehensive understanding of temporal dynamics. It then uses subspace clustering to dynamically identify distinct patterns across data patches. Finally, pattern-specific experts model these unique patterns, delivering tailored predictions for each patch. By explicitly learning and adapting to evolving patterns, TFPS achieves significantly improved forecasting accuracy. Extensive experiments on real-world datasets demonstrate that TFPS outperforms state-of-the-art methods, particularly in long-term forecasting, through its dynamic and pattern-aware learning approach. The data and codes are available: https://github.com/syrGitHub/TFPS.
Key facts. Cited by 23 papers in the map (data through 7 Oct 2026). Builds on 45 of its 75 references that are in the map. First posted on arXiv 13 Oct 2024.
Builds on
45 of 75 references.
References · 45
- 2017Attention Is All You Needcited by 59,932
- 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,204
- 2019PyTorch: An Imperative Style, High-Performance Deep Learning Librarycited by 20,160
- 2017Outrageously Large Neural Networks: The Sparsely-Gated Mixture-of-Experts Layercited by 3,413
- 2021Switch Transformers: Scaling to Trillion Parameter Models with Simple and Efficient Sparsitycited by 3,014
- 2020Informer: Beyond Efficient Transformer for Long Sequence Time-Series Forecastingcited by 2,141
- 2022A Time Series is Worth 64 Words: Long-term Forecasting with Transformerscited by 1,981
- 2021Autoformer: Decomposition Transformers with Auto-Correlation for Long-Term Series Forecastingcited by 1,654
- 2022Are Transformers Effective for Time Series Forecasting?cited by 1,400
- 2022FEDformer: Frequency Enhanced Decomposed Transformer for Long-term Series Forecastingcited by 1,200
- 2022TimesNet: Temporal 2D-Variation Modeling for General Time Series Analysiscited by 1,030
- 2023iTransformer: Inverted Transformers Are Effective for Time Series Forecastingcited by 906
33 more
- 2021GLaM: Efficient Scaling of Language Models with Mixture-of-Expertscited by 890
- 2017Modeling Long- and Short-Term Temporal Patterns with Deep Neural Networkscited by 869
- 2024DeepSeekMoE: Towards Ultimate Expert Specialization in Mixture-of-Experts Language Modelscited by 834
- 2023Time-LLM: Time Series Forecasting by Reprogramming Large Language Modelscited by 680
- 2024Unified Training of Universal Time Series Forecasting Transformerscited by 588
- 2020Learning under Concept Drift: A Reviewcited by 470
- 2024MOMENT: A Family of Open Time-series Foundation Modelscited by 453
- 2022Non-stationary Transformers: Exploring the Stationarity in Time Series Forecastingcited by 385
- 2021FNet: Mixing Tokens with Fourier Transformscited by 343
- 2024MACE: Mass Concept Erasure in Diffusion Modelscited by 298
- 2023From Sparse to Soft Mixtures of Expertscited by 245
- 2024TFB: Towards Comprehensive and Fair Benchmarking of Time Series Forecasting Methodscited by 242
- 2024Timer: Generative Pre-trained Transformers Are Large Time Series Modelscited by 198
- 2023FITS: Modeling Time Series with 10k Parameterscited by 185
- 2023TF-ICON: Diffusion-Based Training-Free Cross-Domain Image Compositioncited by 182
- 2015On Wasserstein Two Sample Testing and Related Families of Nonparametric Testscited by 165
- 2023TSMixer: Lightweight MLP-Mixer Model for Multivariate Time Series Forecastingcited by 136
- 2023Exploring Progress in Multivariate Time Series Forecasting: Comprehensive Benchmarking and Heterogeneity Analysiscited by 113
- 2023Dish-TS: A General Paradigm for Alleviating Distribution Shift in Time Series Forecastingcited by 91
- 2023OneNet: Enhancing Time Series Forecasting Models under Concept Drift by Online Ensemblingcited by 71
- 2024TSLANet: Rethinking Transformers for Time Series Representation Learningcited by 71
- 2024MoE-Mamba: Efficient Selective State Space Models with Mixture of Expertscited by 60
- 2021Person Re-identification Method Based on Color Attack and Joint Defencecited by 56
- 2024Cross-Modality Perturbation Synergy Attack for Person Re-identificationcited by 45
- 2021Eliminate Deviation with Deviation for Data Augmentation and a General Multi-modal Data Learning Methodcited by 42
- 2019Divide and Conquer the Embedding Space for Metric Learningcited by 39
- 2024BlackMamba: Mixture of Experts for State-Space Modelscited by 37
- 2023Mixture-of-Linear-Experts for Long-term Time Series Forecastingcited by 29
- 2023Cross-city Few-Shot Traffic Forecasting via Traffic Pattern Bankcited by 22
- 2023An Accurate and Interpretable Framework for Trustworthy Process Monitoringcited by 18
- 2023TFDNet: Time-Frequency Enhanced Decomposed Network for Long-term Time Series Forecastingcited by 7
- 2024Continual Traffic Forecasting via Mixture of Expertscited by 6
- 2023A Joint Time-frequency Domain Transformer for Multivariate Time Series Forecastingcited by 5
Built on by
Citing papers per year: 2025: 9; 2026: 14.
23 papers.
Sort:
By area:
- 2025Patch-wise Structural Loss for Time Series Forecastingcited by 34
- 2025Out-of-Distribution Generalization in Time Series: A Surveycited by 15
- 2025ASTGI: Adaptive Spatio-Temporal Graph Interactions for Irregular Multivariate Time Series Forecastingcited by 7
- 2025Diffusion-Based Image Editing: An Unforeseen Adversary to Robust Invisible Watermarkscited by 2
- 2026Vanishing Watermarks: Diffusion-Based Image Editing Undermines Robust Invisible Watermarkingcited by 2
- 2025Time Tracker: Mixture-of-Experts-Enhanced Foundation Time Series Forecasting Model with Decoupled Training Pipelinescited by 1
- 2025Beyond All-to-All: Causal-Aligned Transformer with Dynamic Structure Learning for Multivariate Time Series Forecastingcited by 1
- 2025N-BEATS-MOE: N-BEATS with a Mixture-of-Experts Layer for Heterogeneous Time Series Forecastingcited by 1
- 2025SEED: Spectral Entropy-Guided Evaluation of SpatialTemporal Dependencies for Multivariate Time Series Forecastingcited by 1
- 2026Multi-Modal Time Series Prediction via Mixture of Modulated Expertscited by 1
- 2026Editing on the Generative Manifold: A Theoretical and Empirical Study of General Diffusion-Based Image Editing Trade-offscited by 1
- 2025Adapting LLMs to Time Series Forecasting via Temporal Heterogeneity Modeling and Representation Alignmentcited by 0
11 more
- 2026Manifold-Aware Temporal Domain Generalization for Large Language Modelscited by 0
- 2026When Denoising Becomes Unsigning: Theoretical and Empirical Analysis of Watermark Fragility Under Diffusion-Based Image Editingcited by 0
- 2026Editing Away the Evidence: Diffusion-Based Image Manipulation and the Failure Modes of Robust Watermarkingcited by 0
- 2026Dynamic TMoE: A Drift-Aware Dynamic Mixture of Experts Framework for Non-Stationary Time Series Forecastingcited by 0
- 2026On the Controllability-Fidelity Frontier in Diffusion Editingcited by 0
- 2026CausalMoE: A Billion-Scale Multimodal Foundation Model for Granger Causal Discovery with Pattern-Routed Heterogeneous Expertscited by 0
- 2026Towards a Unified Generative Model for Scarce Time Series with Domain Expertscited by 0
- 2026RAVEN: A Regime-Aware Variable-context Expert Network for Financial Time Series Forecastingcited by 0
- 2026Towards Reliable Zero-Shot Crowd Forecasting: Evaluating Time Series Foundation Models for Special Event Pedestrian Forecastingcited by 0
- 2026Which Histories Matter for Time Series Forecasting? Learning Predictive Relevance with Future Supervisioncited by 0
- 2026Aurora-X: Built for Extreme Time Series Forecastingcited by 0
- 2026Aurora-X: Built for Extreme Time Series Forecasting25 Sep 2026
- 2026Which Histories Matter for Time Series Forecasting? Learning Predictive Relevance with Future Supervision24 Aug 2026
- 2026Towards Reliable Zero-Shot Crowd Forecasting: Evaluating Time Series Foundation Models for Special Event Pedestrian Forecasting20 Jul 2026
- 2026RAVEN: A Regime-Aware Variable-context Expert Network for Financial Time Series Forecasting23 Jun 2026
- 2026Towards a Unified Generative Model for Scarce Time Series with Domain Experts13 Jun 2026
- 2026CausalMoE: A Billion-Scale Multimodal Foundation Model for Granger Causal Discovery with Pattern-Routed Heterogeneous Experts11 Jun 2026
- 2026On the Controllability-Fidelity Frontier in Diffusion Editing5 Jun 2026
- 2026Dynamic TMoE: A Drift-Aware Dynamic Mixture of Experts Framework for Non-Stationary Time Series Forecasting20 May 2026
- 2026Editing on the Generative Manifold: A Theoretical and Empirical Study of General Diffusion-Based Image Editing Trade-offs31 Mar 2026
- 2026Editing Away the Evidence: Diffusion-Based Image Manipulation and the Failure Modes of Robust Watermarking13 Mar 2026
- 2026When Denoising Becomes Unsigning: Theoretical and Empirical Analysis of Watermark Fragility Under Diffusion-Based Image Editing5 Mar 2026
- 2026Vanishing Watermarks: Diffusion-Based Image Editing Undermines Robust Invisible Watermarking24 Feb 2026
11 more
- 2026Manifold-Aware Temporal Domain Generalization for Large Language Models12 Feb 2026
- 2026Multi-Modal Time Series Prediction via Mixture of Modulated Experts29 Jan 2026
- 2025SEED: Spectral Entropy-Guided Evaluation of SpatialTemporal Dependencies for Multivariate Time Series Forecasting9 Dec 2025
- 2025Diffusion-Based Image Editing: An Unforeseen Adversary to Robust Invisible Watermarks5 Nov 2025
- 2025ASTGI: Adaptive Spatio-Temporal Graph Interactions for Irregular Multivariate Time Series Forecasting27 Sep 2025
- 2025N-BEATS-MOE: N-BEATS with a Mixture-of-Experts Layer for Heterogeneous Time Series Forecasting10 Aug 2025
- 2025Adapting LLMs to Time Series Forecasting via Temporal Heterogeneity Modeling and Representation Alignment10 Aug 2025
- 2025Beyond All-to-All: Causal-Aligned Transformer with Dynamic Structure Learning for Multivariate Time Series Forecasting22 May 2025
- 2025Time Tracker: Mixture-of-Experts-Enhanced Foundation Time Series Forecasting Model with Decoupled Training Pipelines21 May 2025
- 2025Out-of-Distribution Generalization in Time Series: A Survey18 Mar 2025
- 2025Patch-wise Structural Loss for Time Series Forecasting2 Mar 2025
- 2025Diffusion-Based Image Editing: An Unforeseen Adversary to Robust Invisible Watermarkscited by 2
- 2026Vanishing Watermarks: Diffusion-Based Image Editing Undermines Robust Invisible Watermarkingcited by 2
- 2026Editing on the Generative Manifold: A Theoretical and Empirical Study of General Diffusion-Based Image Editing Trade-offscited by 1
- 2026When Denoising Becomes Unsigning: Theoretical and Empirical Analysis of Watermark Fragility Under Diffusion-Based Image Editingcited by 0
- 2026Editing Away the Evidence: Diffusion-Based Image Manipulation and the Failure Modes of Robust Watermarkingcited by 0
- 2026On the Controllability-Fidelity Frontier in Diffusion Editingcited by 0
- 2026On the Controllability-Fidelity Frontier in Diffusion Editing5 Jun 2026
- 2026Editing on the Generative Manifold: A Theoretical and Empirical Study of General Diffusion-Based Image Editing Trade-offs31 Mar 2026
- 2026Editing Away the Evidence: Diffusion-Based Image Manipulation and the Failure Modes of Robust Watermarking13 Mar 2026
- 2026When Denoising Becomes Unsigning: Theoretical and Empirical Analysis of Watermark Fragility Under Diffusion-Based Image Editing5 Mar 2026
- 2026Vanishing Watermarks: Diffusion-Based Image Editing Undermines Robust Invisible Watermarking24 Feb 2026
- 2025Diffusion-Based Image Editing: An Unforeseen Adversary to Robust Invisible Watermarks5 Nov 2025
- 2025Time Tracker: Mixture-of-Experts-Enhanced Foundation Time Series Forecasting Model with Decoupled Training Pipelinescited by 1
- 2025N-BEATS-MOE: N-BEATS with a Mixture-of-Experts Layer for Heterogeneous Time Series Forecastingcited by 1
- 2026Multi-Modal Time Series Prediction via Mixture of Modulated Expertscited by 1
- 2026Dynamic TMoE: A Drift-Aware Dynamic Mixture of Experts Framework for Non-Stationary Time Series Forecastingcited by 0
- 2026Dynamic TMoE: A Drift-Aware Dynamic Mixture of Experts Framework for Non-Stationary Time Series Forecasting20 May 2026
- 2026Multi-Modal Time Series Prediction via Mixture of Modulated Experts29 Jan 2026
- 2025N-BEATS-MOE: N-BEATS with a Mixture-of-Experts Layer for Heterogeneous Time Series Forecasting10 Aug 2025
- 2025Time Tracker: Mixture-of-Experts-Enhanced Foundation Time Series Forecasting Model with Decoupled Training Pipelines21 May 2025
- 2025Patch-wise Structural Loss for Time Series Forecastingcited by 34
- 2025Out-of-Distribution Generalization in Time Series: A Surveycited by 15
- 2025ASTGI: Adaptive Spatio-Temporal Graph Interactions for Irregular Multivariate Time Series Forecastingcited by 7
- 2025Beyond All-to-All: Causal-Aligned Transformer with Dynamic Structure Learning for Multivariate Time Series Forecastingcited by 1
- 2025SEED: Spectral Entropy-Guided Evaluation of SpatialTemporal Dependencies for Multivariate Time Series Forecastingcited by 1
- 2025Adapting LLMs to Time Series Forecasting via Temporal Heterogeneity Modeling and Representation Alignmentcited by 0
- 2026Manifold-Aware Temporal Domain Generalization for Large Language Modelscited by 0
- 2026CausalMoE: A Billion-Scale Multimodal Foundation Model for Granger Causal Discovery with Pattern-Routed Heterogeneous Expertscited by 0
- 2026Towards a Unified Generative Model for Scarce Time Series with Domain Expertscited by 0
- 2026RAVEN: A Regime-Aware Variable-context Expert Network for Financial Time Series Forecastingcited by 0
- 2026Towards Reliable Zero-Shot Crowd Forecasting: Evaluating Time Series Foundation Models for Special Event Pedestrian Forecastingcited by 0
- 2026Which Histories Matter for Time Series Forecasting? Learning Predictive Relevance with Future Supervisioncited by 0
12 most cited of 13 All 13 (JSON)
- 2026Aurora-X: Built for Extreme Time Series Forecasting25 Sep 2026
- 2026Which Histories Matter for Time Series Forecasting? Learning Predictive Relevance with Future Supervision24 Aug 2026
- 2026Towards Reliable Zero-Shot Crowd Forecasting: Evaluating Time Series Foundation Models for Special Event Pedestrian Forecasting20 Jul 2026
- 2026RAVEN: A Regime-Aware Variable-context Expert Network for Financial Time Series Forecasting23 Jun 2026
- 2026Towards a Unified Generative Model for Scarce Time Series with Domain Experts13 Jun 2026
- 2026CausalMoE: A Billion-Scale Multimodal Foundation Model for Granger Causal Discovery with Pattern-Routed Heterogeneous Experts11 Jun 2026
- 2026Manifold-Aware Temporal Domain Generalization for Large Language Models12 Feb 2026
- 2025SEED: Spectral Entropy-Guided Evaluation of SpatialTemporal Dependencies for Multivariate Time Series Forecasting9 Dec 2025
- 2025ASTGI: Adaptive Spatio-Temporal Graph Interactions for Irregular Multivariate Time Series Forecasting27 Sep 2025
- 2025Adapting LLMs to Time Series Forecasting via Temporal Heterogeneity Modeling and Representation Alignment10 Aug 2025
- 2025Beyond All-to-All: Causal-Aligned Transformer with Dynamic Structure Learning for Multivariate Time Series Forecasting22 May 2025
- 2025Out-of-Distribution Generalization in Time Series: A Survey18 Mar 2025
12 newest of 13 All 13 (JSON)