mlsanityData to 14 Oct 2026

Mixture-of-Linear-Experts for Long-term Time Series Forecasting

arXiv 2312.06786 ↗First posted 11 Dec 2023cs.LG cs.AIcited by 29

Not on the map. Why?

Abstract (arXiv)

Long-term time series forecasting (LTSF) aims to predict future values of a time series given the past values. The current state-of-the-art (SOTA) on this problem is attained in some cases by linear-centric models, which primarily feature a linear mapping layer. However, due to their inherent simplicity, they are not able to adapt their prediction rules to periodic changes in time series patterns. To address this challenge, we propose a Mixture-of-Experts-style augmentation for linear-centric models and propose Mixture-of-Linear-Experts (MoLE). Instead of training a single model, MoLE trains multiple linear-centric models (i.e., experts) and a router model that weighs and mixes their outputs. While the entire framework is trained end-to-end, each expert learns to specialize in a specific temporal pattern, and the router model learns to compose the experts adaptively. Experiments show that MoLE reduces forecasting error of linear-centric models, including DLinear, RLinear, and RMLP, in over 78% of the datasets and settings we evaluated. By using MoLE existing linear-centric models can achieve SOTA LTSF results in 68% of the experiments that PatchTST reports and we compare to, whereas existing single-head linear-centric models achieve SOTA results in only 25% of cases.

Key facts. Cited by 29 papers in the map (data through 14 Oct 2026). Builds on 29 of its 62 references that are in the map. First posted on arXiv 11 Dec 2023.

Builds on

29 of 62 references.

References · 29

  1. 2017Outrageously Large Neural Networks: The Sparsely-Gated Mixture-of-Experts Layercited by 3,509
  2. 2020Informer: Beyond Efficient Transformer for Long Sequence Time-Series Forecastingcited by 2,169
  3. 2022A Time Series is Worth 64 Words: Long-term Forecasting with Transformerscited by 2,039
  4. 2021Autoformer: Decomposition Transformers with Auto-Correlation for Long-Term Series Forecastingcited by 1,676
  5. 2018An Empirical Evaluation of Generic Convolutional and Recurrent Networks for Sequence Modelingcited by 1,476
  6. 2022Are Transformers Effective for Time Series Forecasting?cited by 1,424
  7. 2016On Large-Batch Training for Deep Learning: Generalization Gap and Sharp Minimacited by 1,352
  8. 2022FEDformer: Frequency Enhanced Decomposed Transformer for Long-term Series Forecastingcited by 1,210
  9. 2017Modeling Long- and Short-Term Temporal Patterns with Deep Neural Networkscited by 879
  10. 2021Scaling Vision with Sparse Mixture of Expertscited by 747
  11. 2023Large Language Models Are Zero-Shot Time Series Forecasterscited by 500
  12. 2022Transformers in Time Series: A Surveycited by 497
17 more
  1. 2023Long-term Forecasting with TiDE: Time-series Dense Encodercited by 332
  2. 2023TSMixer: An All-MLP Architecture for Time Series Forecastingcited by 240
  3. 2021Hash Layers For Large Sparse Modelscited by 237
  4. 2023FITS: Modeling Time Series with 10k Parameterscited by 195
  5. 2021Efficient Large Scale Language Modeling with Mixtures of Expertscited by 175
  6. 2022Less Is More: Fast Multivariate Time Series Forecasting with Light Sampling-oriented MLP Structurescited by 160
  7. 2022Spatial-Temporal Identity: A Simple yet Effective Baseline for Multivariate Time Series Forecastingcited by 151
  8. 2023PDFormer: Propagation Delay-Aware Dynamic Long-Range Transformer for Traffic Flow Predictioncited by 137
  9. 2017Conditional Time Series Forecasting with Convolutional Neural Networkscited by 123
  10. 2023EdgeMoE: Empowering Sparse Large Language Models on Mobile Devicescited by 69
  11. 2022Preformer: Predictive Transformer with Multi-Scale Segment-wise Correlations for Long-Term Time Series Forecastingcited by 23
  12. 2018A Memory-Network Based Solution for Multivariate Time-Series Forecastingcited by 22
  13. 2023ST-MLP: A Cascaded Spatio-Temporal Linear Framework with Channel-Independence Strategy for Traffic Forecastingcited by 22
  14. 2023Weather2K: A Multivariate Spatio-Temporal Benchmark Dataset for Meteorological Forecasting Based on Real-Time Observation Data from Ground Weather Stationscited by 18
  15. 2022Spatial Mixture-of-Expertscited by 14
  16. 2022TreeDRNet:A Robust Deep Model for Long Term Time Series Forecastingcited by 4
  17. 2023TLNets: Transformation Learning Networks for long-range time-series predictioncited by 2

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2024: 8; 2025: 10; 2026: 11.

29 papers.

Sort:

By area:

  1. 2024Time-MoE: Billion-Scale Time Series Foundation Models with Mixture of Expertscited by 287
  2. 2024Learning Pattern-Specific Experts for Time Series Forecasting Under Patch-level Distribution Shiftcited by 24
  3. 2025Learning Soft Sparse Shapes for Efficient Time-Series Classificationcited by 10
  4. 2024Disentangled Parameter-Efficient Linear Model for Long-Term Time Series Forecastingcited by 4
  5. 2025Bridging Simplicity and Sophistication using GLinear: A Novel Architecture for Enhanced Time Series Predictioncited by 4
  6. 2024A Survey of Spatio-Temporal EEG data Analysis: from Models to Applicationscited by 3
  7. 2025Rethinking Gating Mechanism in Sparse MoE: Handling Arbitrary Modality Inputs with Confidence-Guided Gatecited by 3
  8. 2025Super-Linear: A Lightweight Pretrained Mixture of Linear Experts for Time Series Forecastingcited by 2
  9. 2024Precise and Efficient Orbit Prediction in LEO with Machine Learning using Exogenous Variablescited by 1
  10. 2024VE: Modeling Multivariate Time Series Correlation with Variate Embeddingcited by 1
  11. 2024LeMoLE: LLM-Enhanced Mixture of Linear Experts for Time Series Forecastingcited by 1
  12. 2025Leveraging External Factors in Household-Level Electrical Consumption Forecasting using Hypernetworkscited by 1
17 more
  1. 2025N-BEATS-MOE: N-BEATS with a Mixture-of-Experts Layer for Heterogeneous Time Series Forecastingcited by 1
  2. 2026Multi-Modal Time Series Prediction via Mixture of Modulated Expertscited by 1
  3. 2024LATST: Are Transformers Necessarily Complex for Time-Series Forecastingcited by 0
  4. 2025Adapting LLMs to Time Series Forecasting via Temporal Heterogeneity Modeling and Representation Alignmentcited by 0
  5. 2025LightSAE: Parameter-Efficient and Heterogeneity-Aware Embedding for IoT Multivariate Time Series Forecastingcited by 0
  6. 2025EMTSF:Extraordinary Mixture of SOTA Models for Time Series Forecastingcited by 0
  7. 2025Naga: Vedic Encoding for Deep State Space Modelscited by 0
  8. 2026MoHETS: Long-term Time Series Forecasting with Mixture-of-Heterogeneous-Expertscited by 0
  9. 2026Deep Time-Series Forecasting in 10 Years: A Surveycited by 0
  10. 2026Temporal Functional Circuits: From Spline Plots to Faithful Explanations in KAN Forecastingcited by 0
  11. 2026TSNN: A Non-parametric and Interpretable Framework for Traffic Time Series Forecastingcited by 0
  12. 2026Fast Training of Mixture-of-Experts for Time Series Forecasting via Expert Loss Integrationcited by 0
  13. 2026Dynamic TMoE: A Drift-Aware Dynamic Mixture of Experts Framework for Non-Stationary Time Series Forecastingcited by 0
  14. 2026Modular Foundation Models for Time-Series Perception in Digital Twinscited by 0
  15. 2026MoFE: A Novel Mixture-of-Experts Framework with Fourier Neural Operators for Cryptocurrency Forecastingcited by 0
  16. 2026Structure Aware Neural Architecture Search for Mixture of Expertscited by 0
  17. 2026Rollcast: Proper-Score Gated Rolling Anchors for Adaptive Probabilistic Time-Series Forecastingcited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2026AME-TS: Anchored Mixture-of-Experts for Time Series Forecasting
  2. 2025MoFE-Time: Mixture of Frequency Domain Experts for Time-Series Forecasting Models
  3. 2026FM-LLM: A frequency-enhanced mixture-of-experts framework for adapting LLMs to time series forecasting
  4. 2026Seg-MoE: Multi-Resolution Segment-wise Mixture-of-Experts for Time Series Forecasting Transformers
  5. 2026WaveMoE: A Wavelet-Enhanced Mixture-of-Experts Foundation Model for Time Series Forecasting
  6. 2025Time Tracker: Mixture-of-Experts-Enhanced Foundation Time Series Forecasting Model with Decoupled Training Pipelines
  7. 2025MoGU: Mixture-of-Gaussians with Uncertainty-based Gating for Time Series Forecasting
  8. 2025Small but Mighty: Enhancing Time Series Forecasting with Lightweight LLMs