mlsanityData to 7 Oct 2026

xLSTM-Mixer: Multivariate Time Series Forecasting by Mixing via Scalar Memories

arXiv 2410.16928 ↗First posted 22 Oct 2024cs.LGcited by 18

Not on the map. Why?

Abstract (arXiv)

Time series data is prevalent across numerous fields, necessitating the development of robust and accurate forecasting models. Capturing patterns both within and between temporal and multivariate components is crucial for reliable predictions. We introduce xLSTM-Mixer, a model designed to effectively integrate temporal sequences, joint time-variate information, and multiple perspectives for robust forecasting. Our approach begins with a linear forecast shared across variates, which is then refined by xLSTM blocks. They serve as key elements for modeling the complex dynamics of challenging time series data. xLSTM-Mixer ultimately reconciles two distinct views to produce the final forecast. Our extensive evaluations demonstrate its superior long-term forecasting performance compared to recent state-of-the-art methods while requiring very little memory. A thorough model analysis provides further insights into its key components and confirms its robustness and effectiveness. This work contributes to the resurgence of recurrent models in forecasting by combining them, for the first time, with mixing architectures.

Key facts. Cited by 18 papers in the map (data through 7 Oct 2026). Builds on 62 of its 88 references that are in the map. First posted on arXiv 22 Oct 2024.

Builds on

62 of 88 references.

References · 62

  1. 2017Attention Is All You Needcited by 59,932
  2. 2019PyTorch: An Imperative Style, High-Performance Deep Learning Librarycited by 20,160
  3. 2016XGBoost: A Scalable Tree Boosting Systemcited by 5,120
  4. 2023Mamba: Linear-Time Sequence Modeling with Selective State Spacescited by 4,616
  5. 2014Learning Phrase Representations using RNN Encoder-Decoder for Statistical Machine Translationcited by 4,016
  6. 2021Prefix-Tuning: Optimizing Continuous Prompts for Generationcited by 3,878
  7. 2021The Power of Scale for Parameter-Efficient Prompt Tuningcited by 3,769
  8. 2019Optuna: A Next-generation Hyperparameter Optimization Frameworkcited by 2,455
  9. 2021Efficiently Modeling Long Sequences with Structured State Spacescited by 2,424
  10. 2020Informer: Beyond Efficient Transformer for Long Sequence Time-Series Forecastingcited by 2,141
  11. 2022A Time Series is Worth 64 Words: Long-term Forecasting with Transformerscited by 1,981
  12. 2020Reformer: The Efficient Transformercited by 1,658
38 more
  1. 2021Autoformer: Decomposition Transformers with Auto-Correlation for Long-Term Series Forecastingcited by 1,654
  2. 2021MLP-Mixer: An all-MLP Architecture for Visioncited by 1,441
  3. 2022Are Transformers Effective for Time Series Forecasting?cited by 1,400
  4. 2022FEDformer: Frequency Enhanced Decomposed Transformer for Long-term Series Forecastingcited by 1,200
  5. 2022TimesNet: Temporal 2D-Variation Modeling for General Time Series Analysiscited by 1,030
  6. 2017DeepAR: Probabilistic Forecasting with Autoregressive Recurrent Networkscited by 1,004
  7. 2023iTransformer: Inverted Transformers Are Effective for Time Series Forecastingcited by 906
  8. 2017Modeling Long- and Short-Term Temporal Patterns with Deep Neural Networkscited by 869
  9. 2017A Survey on Multi-Task Learningcited by 838
  10. 2019Temporal Fusion Transformers for Interpretable Multi-horizon Time Series Forecastingcited by 748
  11. 2019N-BEATS: Neural basis expansion analysis for interpretable time series forecastingcited by 744
  12. 2024Chronos: Learning the Language of Time Seriescited by 712
  13. 2023Time-LLM: Time Series Forecasting by Reprogramming Large Language Modelscited by 680
  14. 2024Unified Training of Universal Time Series Forecasting Transformerscited by 588
  15. 2020Captum: A unified and generic model interpretability library for PyTorchcited by 542
  16. 2020A Survey of Convolutional Neural Networks: Analysis, Applications, and Prospectscited by 396
  17. 2022Non-stationary Transformers: Exploring the Stationarity in Time Series Forecastingcited by 385
  18. 2019ROCKET: Exceptionally fast and accurate time series classification using random convolutional kernelscited by 336
  19. 2024TimeMixer: Decomposable Multiscale Mixing for Time Series Forecastingcited by 333
  20. 2023Long-term Forecasting with TiDE: Time-series Dense Encodercited by 327
  21. 2024xLSTM: Extended Long Short-Term Memorycited by 320
  22. 2015LSTM: A Search Space Odysseycited by 315
  23. 2023TSMixer: An All-MLP Architecture for Time Series Forecastingcited by 232
  24. 2016Temporal Convolutional Networks: A Unified Approach to Action Segmentationcited by 199
  25. 2024Deep Time Series Models: A Comprehensive Survey and Benchmarkcited by 191
  26. 2024GIFT-Eval: A Benchmark For General Time Series Forecasting Model Evaluationcited by 188
  27. 2023TEMPO: Prompt-based Generative Pre-trained Transformer for Time Series Forecastingcited by 170
  28. 2023TEST: Text Prototype Aligned Embedding to Activate LLM's Ability for Time Seriescited by 165
  29. 2022ETSformer: Exponential Smoothing Transformers for Time-series Forecastingcited by 146
  30. 2024TimeMixer++: A General Time Series Pattern Machine for Universal Predictive Analysiscited by 113
  31. 2024Is Mamba Effective for Time Series Forecasting?cited by 110
  32. 2024CycleNet: Enhancing Time Series Forecasting through Modeling Periodic Patternscited by 102
  33. 2020Normalization Techniques in Training DNNs: Methodology, Analysis and Applicationcited by 90
  34. 2024Timer-XL: Long-Context Transformers for Unified Time Series Forecastingcited by 83
  35. 2022Flowformer: Linearizing Transformers with Conservation Flowscited by 69
  36. 2024Mamba-360: Survey of State Space Models as Transformer Alternative for Long Sequence Modelling: Methods, Applications, and Challengescited by 63
  37. 2023How Many Demonstrations Do You Need for In-context Learning?cited by 52
  38. 2023LightTS: Lightweight Time Series Classification with Adaptive Ensemble Distillation -- Extended Versioncited by 49

50 most cited of 62 All 62 (JSON)

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2025: 13; 2026: 5.

18 papers.

Sort:

By area:

  1. 2025TiRex: Zero-Shot Forecasting Across Long and Short Horizons with Enhanced In-Context Learningcited by 107
  2. 2025xLSTM 7B: A Recurrent LLM for Fast and Efficient Inferencecited by 17
  3. 2025Tiled Flash Linear Attention: More Efficient Linear RNN and xLSTM Kernelscited by 12
  4. 2025There are no Champions in Supervised Long-Term Time Series Forecastingcited by 8
  5. 2025Learning General Causal Structures with Hidden Dynamic Process for Climate Analysiscited by 6
  6. 2025QuAnTS: Question Answering on Time Seriescited by 5
  7. 2025Load Forecasting for Households and Energy Communities: Are Deep Learning Models Worth the Effort?cited by 1
  8. 2025Exploring Neural Granger Causality with xLSTMs: Unveiling Temporal Dependencies in Complex Datacited by 1
  9. 2025Advanced Hybrid Transformer LSTM Technique with Attention and TS Mixer for Drilling Rate of Penetration Predictioncited by 1
  10. 2025X-TRACK: Physics-Aware xLSTM for Realistic Vehicle Trajectory Predictioncited by 1
  11. 2025TimePerceiver: An Encoder-Decoder Framework for Generalized Time-Series Forecastingcited by 1
  12. 2026CSRA: Controlled Spectral Residual Augmentation for Robust Sepsis Predictioncited by 1
6 more
  1. 2025StoxLSTM: A Stochastic Extended Long Short-Term Memory Network for Time Series Forecastingcited by 0
  2. 2025An overview of neural architectures for self-supervised audio representation learning from masked spectrogramscited by 0
  3. 2026MLOW: Interpretable Low-Rank Frequency Magnitude Decomposition of Multiple Effects for Time Series Forecastingcited by 0
  4. 2026Benchmarking Transformer and xLSTM for Time-Series Forecasting of Heat Consumptioncited by 0
  5. 2026TSSM: Triaxial State Space Model for Global Station Weather Forecasting with Temporal-Variable-Historical Modelingcited by 0
  6. 2026Reliable Virtual Sensing: A Multi-Domain Benchmark for Robustness Under Sensor Failurescited by 0