mlsanityData to 7 Oct 2026

VE: Modeling Multivariate Time Series Correlation with Variate Embedding

arXiv 2409.06169 ↗First posted 10 Sep 2024cs.LGcited by 1

On the map: Mixture-of-experts → MoE for time series and graphs

Abstract (arXiv)

Multivariate time series forecasting relies on accurately capturing the correlations among variates. Current channel-independent (CI) models and models with a CI final projection layer are unable to capture these dependencies. In this paper, we present the variate embedding (VE) pipeline, which learns a unique and consistent embedding for each variate and combines it with Mixture of Experts (MoE) and Low-Rank Adaptation (LoRA) techniques to enhance forecasting performance while controlling parameter size. The VE pipeline can be integrated into any model with a CI final projection layer to improve multivariate forecasting. The learned VE effectively groups variates with similar temporal patterns and separates those with low correlations. The effectiveness of the VE pipeline is demonstrated through experiments on four widely-used datasets. The code is available at: https://github.com/swang-song/VE.

Key facts. Cited by 1 paper in the map (data through 7 Oct 2026). Placed in MoE for time series and graphs, one of 12 directions in Mixture-of-experts. Builds on 26 of its 29 references that are in the map. First posted on arXiv 10 Sep 2024.

Builds on

26 of 29 references.

References · 26

  1. 2017Attention Is All You Needcited by 59,932
  2. 2021LoRA: Low-Rank Adaptation of Large Language Modelscited by 14,084
  3. 2017Outrageously Large Neural Networks: The Sparsely-Gated Mixture-of-Experts Layercited by 3,413
  4. 2022A Time Series is Worth 64 Words: Long-term Forecasting with Transformerscited by 1,981
  5. 2016WaveNet: A Generative Model for Raw Audiocited by 1,710
  6. 2021Autoformer: Decomposition Transformers with Auto-Correlation for Long-Term Series Forecastingcited by 1,654
  7. 2018An Empirical Evaluation of Generic Convolutional and Recurrent Networks for Sequence Modelingcited by 1,447
  8. 2022Are Transformers Effective for Time Series Forecasting?cited by 1,400
  9. 2022FEDformer: Frequency Enhanced Decomposed Transformer for Long-term Series Forecastingcited by 1,200
  10. 2017DeepAR: Probabilistic Forecasting with Autoregressive Recurrent Networkscited by 1,004
  11. 2023iTransformer: Inverted Transformers Are Effective for Time Series Forecastingcited by 906
  12. 2017Modeling Long- and Short-Term Temporal Patterns with Deep Neural Networkscited by 869
14 more
  1. 2023Time-LLM: Time Series Forecasting by Reprogramming Large Language Modelscited by 680
  2. 2023A decoder-only foundation model for time-series forecastingcited by 650
  3. 2023Large Language Models Are Zero-Shot Time Series Forecasterscited by 491
  4. 2021SCINet: Time Series Modeling and Forecasting with Sample Convolution and Interactioncited by 335
  5. 2023Long-term Forecasting with TiDE: Time-series Dense Encodercited by 327
  6. 2023TSMixer: An All-MLP Architecture for Time Series Forecastingcited by 232
  7. 2017A Multi-Horizon Quantile Recurrent Forecastercited by 211
  8. 2023FITS: Modeling Time Series with 10k Parameterscited by 185
  9. 2023Frequency-domain MLPs are More Effective Learners in Time Series Forecastingcited by 170
  10. 2023Koopa: Learning Non-stationary Time Series Dynamics with Koopman Predictorscited by 110
  11. 2024SparseTSF: Modeling Long-term Time Series Forecasting with 1k Parameterscited by 103
  12. 2024SAMformer: Unlocking the Potential of Transformers in Time Series Forecasting with Sharpness-Aware Minimization and Channel-Wise Attentioncited by 43
  13. 2023BasisFormer: Attention-based Time Series Forecasting with Learnable and Interpretable Basiscited by 41
  14. 2023Mixture-of-Linear-Experts for Long-term Time Series Forecastingcited by 29

Built on by

Cited by year

Citing papers per year: 2025: 1.

1 paper.

  1. 2025LightSAE: Parameter-Efficient and Heterogeneity-Aware Embedding for IoT Multivariate Time Series Forecastingcited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2026vLinear: A Powerful Linear Model for Multivariate Time Series Forecasting
  2. 2024CMamba: Channel Correlation Enhanced State Space Models for Multivariate Time Series Forecasting
  3. 2024TimeCMA: Towards LLM-Empowered Multivariate Time Series Forecasting via Cross-Modality Alignment
  4. 2023LLM4TS: Aligning Pre-Trained LLMs as Data-Efficient Time-Series Forecasters
  5. 2024Are KANs Effective for Multivariate Time Series Forecasting?
  6. 2025Time Tracker: Mixture-of-Experts-Enhanced Foundation Time Series Forecasting Model with Decoupled Training Pipelines
  7. 2025Small but Mighty: Enhancing Time Series Forecasting with Lightweight LLMs
  8. 2025TimePro: Efficient Multivariate Long-term Time Series Forecasting with Variable- and Time-Aware Hyper-state