# xLSTM-Mixer: Multivariate Time Series Forecasting by Mixing via Scalar Memories

arXiv: https://arxiv.org/abs/2410.16928 · HTML: https://mlsanity.com/paper/2410.16928/

## Key facts

- Cited by 19 papers in the map (data through 14 Oct 2026).
- Builds on 62 of its 88 references that are in the map.
- First posted on arXiv 22 Oct 2024.

## Builds on (most cited first, 30 of 62)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [PyTorch: An Imperative Style, High-Performance Deep Learning Library](https://mlsanity.com/paper/1912.01703/) (2019)
- [XGBoost: A Scalable Tree Boosting System](https://mlsanity.com/paper/1603.02754/) (2016)
- [Mamba: Linear-Time Sequence Modeling with Selective State Spaces](https://mlsanity.com/paper/2312.00752/) (2023)
- [Learning Phrase Representations using RNN Encoder-Decoder for Statistical Machine Translation](https://mlsanity.com/paper/1406.1078/) (2014)
- [Prefix-Tuning: Optimizing Continuous Prompts for Generation](https://mlsanity.com/paper/2101.00190/) (2021)
- [The Power of Scale for Parameter-Efficient Prompt Tuning](https://mlsanity.com/paper/2104.08691/) (2021)
- [Optuna: A Next-generation Hyperparameter Optimization Framework](https://mlsanity.com/paper/1907.10902/) (2019)
- [Efficiently Modeling Long Sequences with Structured State Spaces](https://mlsanity.com/paper/2111.00396/) (2021)
- [Informer: Beyond Efficient Transformer for Long Sequence Time-Series Forecasting](https://mlsanity.com/paper/2012.07436/) (2020)
- [A Time Series is Worth 64 Words: Long-term Forecasting with Transformers](https://mlsanity.com/paper/2211.14730/) (2022)
- [Autoformer: Decomposition Transformers with Auto-Correlation for Long-Term Series Forecasting](https://mlsanity.com/paper/2106.13008/) (2021)
- [Reformer: The Efficient Transformer](https://mlsanity.com/paper/2001.04451/) (2020)
- [MLP-Mixer: An all-MLP Architecture for Vision](https://mlsanity.com/paper/2105.01601/) (2021)
- [Are Transformers Effective for Time Series Forecasting?](https://mlsanity.com/paper/2205.13504/) (2022)
- [FEDformer: Frequency Enhanced Decomposed Transformer for Long-term Series Forecasting](https://mlsanity.com/paper/2201.12740/) (2022)
- [TimesNet: Temporal 2D-Variation Modeling for General Time Series Analysis](https://mlsanity.com/paper/2210.02186/) (2022)
- [DeepAR: Probabilistic Forecasting with Autoregressive Recurrent Networks](https://mlsanity.com/paper/1704.04110/) (2017)
- [iTransformer: Inverted Transformers Are Effective for Time Series Forecasting](https://mlsanity.com/paper/2310.06625/) (2023)
- [Modeling Long- and Short-Term Temporal Patterns with Deep Neural Networks](https://mlsanity.com/paper/1703.07015/) (2017)
- [A Survey on Multi-Task Learning](https://mlsanity.com/paper/1707.08114/) (2017)
- [Temporal Fusion Transformers for Interpretable Multi-horizon Time Series Forecasting](https://mlsanity.com/paper/1912.09363/) (2019)
- [Chronos: Learning the Language of Time Series](https://mlsanity.com/paper/2403.07815/) (2024)
- [N-BEATS: Neural basis expansion analysis for interpretable time series forecasting](https://mlsanity.com/paper/1905.10437/) (2019)
- [Time-LLM: Time Series Forecasting by Reprogramming Large Language Models](https://mlsanity.com/paper/2310.01728/) (2023)
- [Unified Training of Universal Time Series Forecasting Transformers](https://mlsanity.com/paper/2402.02592/) (2024)
- [Captum: A unified and generic model interpretability library for PyTorch](https://mlsanity.com/paper/2009.07896/) (2020)
- [A Survey of Convolutional Neural Networks: Analysis, Applications, and Prospects](https://mlsanity.com/paper/2004.02806/) (2020)
- [Non-stationary Transformers: Exploring the Stationarity in Time Series Forecasting](https://mlsanity.com/paper/2205.14415/) (2022)
- [TimeMixer: Decomposable Multiscale Mixing for Time Series Forecasting](https://mlsanity.com/paper/2405.14616/) (2024)

## Built on by (most cited first, 19 of 19)

- [TiRex: Zero-Shot Forecasting Across Long and Short Horizons with Enhanced In-Context Learning](https://mlsanity.com/paper/2505.23719/) (2025)
- [xLSTM 7B: A Recurrent LLM for Fast and Efficient Inference](https://mlsanity.com/paper/2503.13427/) (2025)
- [Tiled Flash Linear Attention: More Efficient Linear RNN and xLSTM Kernels](https://mlsanity.com/paper/2503.14376/) (2025)
- [There are no Champions in Supervised Long-Term Time Series Forecasting](https://mlsanity.com/paper/2502.14045/) (2025)
- [Learning General Causal Structures with Hidden Dynamic Process for Climate Analysis](https://mlsanity.com/paper/2501.12500/) (2025)
- [QuAnTS: Question Answering on Time Series](https://mlsanity.com/paper/2511.05124/) (2025)
- [Load Forecasting for Households and Energy Communities: Are Deep Learning Models Worth the Effort?](https://mlsanity.com/paper/2501.05000/) (2025)
- [Exploring Neural Granger Causality with xLSTMs: Unveiling Temporal Dependencies in Complex Data](https://mlsanity.com/paper/2502.09981/) (2025)
- [Advanced Hybrid Transformer LSTM Technique with Attention and TS Mixer for Drilling Rate of Penetration Prediction](https://mlsanity.com/paper/2508.05210/) (2025)
- [X-TRACK: Physics-Aware xLSTM for Realistic Vehicle Trajectory Prediction](https://mlsanity.com/paper/2511.00266/) (2025)
- [TimePerceiver: An Encoder-Decoder Framework for Generalized Time-Series Forecasting](https://mlsanity.com/paper/2512.22550/) (2025)
- [CSRA: Controlled Spectral Residual Augmentation for Robust Sepsis Prediction](https://mlsanity.com/paper/2604.14532/) (2026)
- [StoxLSTM: A Stochastic Extended Long Short-Term Memory Network for Time Series Forecasting](https://mlsanity.com/paper/2509.01187/) (2025)
- [An overview of neural architectures for self-supervised audio representation learning from masked spectrograms](https://mlsanity.com/paper/2509.18691/) (2025)
- [MLOW: Interpretable Low-Rank Frequency Magnitude Decomposition of Multiple Effects for Time Series Forecasting](https://mlsanity.com/paper/2603.18432/) (2026)
- [Benchmarking Transformer and xLSTM for Time-Series Forecasting of Heat Consumption](https://mlsanity.com/paper/2605.09722/) (2026)
- [TSSM: Triaxial State Space Model for Global Station Weather Forecasting with Temporal-Variable-Historical Modeling](https://mlsanity.com/paper/2607.13101/) (2026)
- [Reliable Virtual Sensing: A Multi-Domain Benchmark for Robustness Under Sensor Failures](https://mlsanity.com/paper/2609.18396/) (2026)
- [HART: Hybrid Attention-RWKV Time series Foundation Model](https://mlsanity.com/paper/2610.12883/) (2026)

## Abstract (arXiv)

Time series data is prevalent across numerous fields, necessitating the development of robust and accurate forecasting models. Capturing patterns both within and between temporal and multivariate components is crucial for reliable predictions. We introduce xLSTM-Mixer, a model designed to effectively integrate temporal sequences, joint time-variate information, and multiple perspectives for robust forecasting. Our approach begins with a linear forecast shared across variates, which is then refined by xLSTM blocks. They serve as key elements for modeling the complex dynamics of challenging time series data. xLSTM-Mixer ultimately reconciles two distinct views to produce the final forecast. Our extensive evaluations demonstrate its superior long-term forecasting performance compared to recent state-of-the-art methods while requiring very little memory. A thorough model analysis provides further insights into its key components and confirms its robustness and effectiveness. This work contributes to the resurgence of recurrent models in forecasting by combining them, for the first time, with mixing architectures.