# Mixture-of-Linear-Experts for Long-term Time Series Forecasting

arXiv: https://arxiv.org/abs/2312.06786 · HTML: https://mlsanity.com/paper/2312.06786/

## Key facts

- Cited by 29 papers in the map (data through 14 Oct 2026).
- Builds on 29 of its 62 references that are in the map.
- First posted on arXiv 11 Dec 2023.

## Builds on (most cited first, 29 of 29)

- [Outrageously Large Neural Networks: The Sparsely-Gated Mixture-of-Experts Layer](https://mlsanity.com/paper/1701.06538/) (2017)
- [Informer: Beyond Efficient Transformer for Long Sequence Time-Series Forecasting](https://mlsanity.com/paper/2012.07436/) (2020)
- [A Time Series is Worth 64 Words: Long-term Forecasting with Transformers](https://mlsanity.com/paper/2211.14730/) (2022)
- [Autoformer: Decomposition Transformers with Auto-Correlation for Long-Term Series Forecasting](https://mlsanity.com/paper/2106.13008/) (2021)
- [An Empirical Evaluation of Generic Convolutional and Recurrent Networks for Sequence Modeling](https://mlsanity.com/paper/1803.01271/) (2018)
- [Are Transformers Effective for Time Series Forecasting?](https://mlsanity.com/paper/2205.13504/) (2022)
- [On Large-Batch Training for Deep Learning: Generalization Gap and Sharp Minima](https://mlsanity.com/paper/1609.04836/) (2016)
- [FEDformer: Frequency Enhanced Decomposed Transformer for Long-term Series Forecasting](https://mlsanity.com/paper/2201.12740/) (2022)
- [Modeling Long- and Short-Term Temporal Patterns with Deep Neural Networks](https://mlsanity.com/paper/1703.07015/) (2017)
- [Scaling Vision with Sparse Mixture of Experts](https://mlsanity.com/paper/2106.05974/) (2021)
- [Large Language Models Are Zero-Shot Time Series Forecasters](https://mlsanity.com/paper/2310.07820/) (2023)
- [Transformers in Time Series: A Survey](https://mlsanity.com/paper/2202.07125/) (2022)
- [Long-term Forecasting with TiDE: Time-series Dense Encoder](https://mlsanity.com/paper/2304.08424/) (2023)
- [TSMixer: An All-MLP Architecture for Time Series Forecasting](https://mlsanity.com/paper/2303.06053/) (2023)
- [Hash Layers For Large Sparse Models](https://mlsanity.com/paper/2106.04426/) (2021)
- [FITS: Modeling Time Series with 10k Parameters](https://mlsanity.com/paper/2307.03756/) (2023)
- [Efficient Large Scale Language Modeling with Mixtures of Experts](https://mlsanity.com/paper/2112.10684/) (2021)
- [Less Is More: Fast Multivariate Time Series Forecasting with Light Sampling-oriented MLP Structures](https://mlsanity.com/paper/2207.01186/) (2022)
- [Spatial-Temporal Identity: A Simple yet Effective Baseline for Multivariate Time Series Forecasting](https://mlsanity.com/paper/2208.05233/) (2022)
- [PDFormer: Propagation Delay-Aware Dynamic Long-Range Transformer for Traffic Flow Prediction](https://mlsanity.com/paper/2301.07945/) (2023)
- [Conditional Time Series Forecasting with Convolutional Neural Networks](https://mlsanity.com/paper/1703.04691/) (2017)
- [EdgeMoE: Empowering Sparse Large Language Models on Mobile Devices](https://mlsanity.com/paper/2308.14352/) (2023)
- [Preformer: Predictive Transformer with Multi-Scale Segment-wise Correlations for Long-Term Time Series Forecasting](https://mlsanity.com/paper/2202.11356/) (2022)
- [A Memory-Network Based Solution for Multivariate Time-Series Forecasting](https://mlsanity.com/paper/1809.02105/) (2018)
- [ST-MLP: A Cascaded Spatio-Temporal Linear Framework with Channel-Independence Strategy for Traffic Forecasting](https://mlsanity.com/paper/2308.07496/) (2023)
- [Weather2K: A Multivariate Spatio-Temporal Benchmark Dataset for Meteorological Forecasting Based on Real-Time Observation Data from Ground Weather Stations](https://mlsanity.com/paper/2302.10493/) (2023)
- [Spatial Mixture-of-Experts](https://mlsanity.com/paper/2211.13491/) (2022)
- [TreeDRNet:A Robust Deep Model for Long Term Time Series Forecasting](https://mlsanity.com/paper/2206.12106/) (2022)
- [TLNets: Transformation Learning Networks for long-range time-series prediction](https://mlsanity.com/paper/2305.15770/) (2023)

## Built on by (most cited first, 29 of 29)

- [Time-MoE: Billion-Scale Time Series Foundation Models with Mixture of Experts](https://mlsanity.com/paper/2409.16040/) (2024)
- [Learning Pattern-Specific Experts for Time Series Forecasting Under Patch-level Distribution Shift](https://mlsanity.com/paper/2410.09836/) (2024)
- [Learning Soft Sparse Shapes for Efficient Time-Series Classification](https://mlsanity.com/paper/2505.06892/) (2025)
- [Disentangled Parameter-Efficient Linear Model for Long-Term Time Series Forecasting](https://mlsanity.com/paper/2411.17257/) (2024)
- [Bridging Simplicity and Sophistication using GLinear: A Novel Architecture for Enhanced Time Series Prediction](https://mlsanity.com/paper/2501.01087/) (2025)
- [A Survey of Spatio-Temporal EEG data Analysis: from Models to Applications](https://mlsanity.com/paper/2410.08224/) (2024)
- [Rethinking Gating Mechanism in Sparse MoE: Handling Arbitrary Modality Inputs with Confidence-Guided Gate](https://mlsanity.com/paper/2505.19525/) (2025)
- [Super-Linear: A Lightweight Pretrained Mixture of Linear Experts for Time Series Forecasting](https://mlsanity.com/paper/2509.15105/) (2025)
- [Precise and Efficient Orbit Prediction in LEO with Machine Learning using Exogenous Variables](https://mlsanity.com/paper/2407.11026/) (2024)
- [VE: Modeling Multivariate Time Series Correlation with Variate Embedding](https://mlsanity.com/paper/2409.06169/) (2024)
- [LeMoLE: LLM-Enhanced Mixture of Linear Experts for Time Series Forecasting](https://mlsanity.com/paper/2412.00053/) (2024)
- [Leveraging External Factors in Household-Level Electrical Consumption Forecasting using Hypernetworks](https://mlsanity.com/paper/2506.14472/) (2025)
- [N-BEATS-MOE: N-BEATS with a Mixture-of-Experts Layer for Heterogeneous Time Series Forecasting](https://mlsanity.com/paper/2508.07490/) (2025)
- [Multi-Modal Time Series Prediction via Mixture of Modulated Experts](https://mlsanity.com/paper/2601.21547/) (2026)
- [LATST: Are Transformers Necessarily Complex for Time-Series Forecasting](https://mlsanity.com/paper/2410.23749/) (2024)
- [Adapting LLMs to Time Series Forecasting via Temporal Heterogeneity Modeling and Representation Alignment](https://mlsanity.com/paper/2508.07195/) (2025)
- [LightSAE: Parameter-Efficient and Heterogeneity-Aware Embedding for IoT Multivariate Time Series Forecasting](https://mlsanity.com/paper/2510.10465/) (2025)
- [EMTSF:Extraordinary Mixture of SOTA Models for Time Series Forecasting](https://mlsanity.com/paper/2510.23396/) (2025)
- [Naga: Vedic Encoding for Deep State Space Models](https://mlsanity.com/paper/2511.13510/) (2025)
- [MoHETS: Long-term Time Series Forecasting with Mixture-of-Heterogeneous-Experts](https://mlsanity.com/paper/2601.21866/) (2026)
- [Deep Time-Series Forecasting in 10 Years: A Survey](https://mlsanity.com/paper/2603.19899/) (2026)
- [Temporal Functional Circuits: From Spline Plots to Faithful Explanations in KAN Forecasting](https://mlsanity.com/paper/2605.05685/) (2026)
- [TSNN: A Non-parametric and Interpretable Framework for Traffic Time Series Forecasting](https://mlsanity.com/paper/2605.09208/) (2026)
- [Fast Training of Mixture-of-Experts for Time Series Forecasting via Expert Loss Integration](https://mlsanity.com/paper/2605.10330/) (2026)
- [Dynamic TMoE: A Drift-Aware Dynamic Mixture of Experts Framework for Non-Stationary Time Series Forecasting](https://mlsanity.com/paper/2605.20678/) (2026)
- [Modular Foundation Models for Time-Series Perception in Digital Twins](https://mlsanity.com/paper/2607.03585/) (2026)
- [MoFE: A Novel Mixture-of-Experts Framework with Fourier Neural Operators for Cryptocurrency Forecasting](https://mlsanity.com/paper/2608.17342/) (2026)
- [Structure Aware Neural Architecture Search for Mixture of Experts](https://mlsanity.com/paper/2608.29817/) (2026)
- [Rollcast: Proper-Score Gated Rolling Anchors for Adaptive Probabilistic Time-Series Forecasting](https://mlsanity.com/paper/2609.05561/) (2026)

## Abstract (arXiv)

Long-term time series forecasting (LTSF) aims to predict future values of a time series given the past values. The current state-of-the-art (SOTA) on this problem is attained in some cases by linear-centric models, which primarily feature a linear mapping layer. However, due to their inherent simplicity, they are not able to adapt their prediction rules to periodic changes in time series patterns. To address this challenge, we propose a Mixture-of-Experts-style augmentation for linear-centric models and propose Mixture-of-Linear-Experts (MoLE). Instead of training a single model, MoLE trains multiple linear-centric models (i.e., experts) and a router model that weighs and mixes their outputs. While the entire framework is trained end-to-end, each expert learns to specialize in a specific temporal pattern, and the router model learns to compose the experts adaptively. Experiments show that MoLE reduces forecasting error of linear-centric models, including DLinear, RLinear, and RMLP, in over 78% of the datasets and settings we evaluated. By using MoLE existing linear-centric models can achieve SOTA LTSF results in 68% of the experiments that PatchTST reports and we compare to, whereas existing single-head linear-centric models achieve SOTA results in only 25% of cases.