# TSMixer: Lightweight MLP-Mixer Model for Multivariate Time Series Forecasting

arXiv: https://arxiv.org/abs/2306.09364 · HTML: https://mlsanity.com/paper/2306.09364/

## Key facts

- Cited by 139 papers in the map (data through 14 Oct 2026).
- Builds on 29 of its 43 references that are in the map.
- First posted on arXiv 14 Jun 2023.

## Builds on (most cited first, 29 of 29)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [Representation Learning with Contrastive Predictive Coding](https://mlsanity.com/paper/1807.03748/) (2018)
- [Layer Normalization](https://mlsanity.com/paper/1607.06450/) (2016)
- [Neural Machine Translation by Jointly Learning to Align and Translate](https://mlsanity.com/paper/1409.0473/) (2014)
- [On the Opportunities and Risks of Foundation Models](https://mlsanity.com/paper/2108.07258/) (2021)
- [Efficiently Modeling Long Sequences with Structured State Spaces](https://mlsanity.com/paper/2111.00396/) (2021)
- [BEiT: BERT Pre-Training of Image Transformers](https://mlsanity.com/paper/2106.08254/) (2021)
- [Informer: Beyond Efficient Transformer for Long Sequence Time-Series Forecasting](https://mlsanity.com/paper/2012.07436/) (2020)
- [A Time Series is Worth 64 Words: Long-term Forecasting with Transformers](https://mlsanity.com/paper/2211.14730/) (2022)
- [Autoformer: Decomposition Transformers with Auto-Correlation for Long-Term Series Forecasting](https://mlsanity.com/paper/2106.13008/) (2021)
- [MLP-Mixer: An all-MLP Architecture for Vision](https://mlsanity.com/paper/2105.01601/) (2021)
- [Are Transformers Effective for Time Series Forecasting?](https://mlsanity.com/paper/2205.13504/) (2022)
- [FEDformer: Frequency Enhanced Decomposed Transformer for Long-term Series Forecasting](https://mlsanity.com/paper/2201.12740/) (2022)
- [TS2Vec: Towards Universal Representation of Time Series](https://mlsanity.com/paper/2106.10466/) (2021)
- [Pay Attention to MLPs](https://mlsanity.com/paper/2105.08050/) (2021)
- [Time-Series Representation Learning via Temporal and Contextual Contrasting](https://mlsanity.com/paper/2106.14112/) (2021)
- [ResMLP: Feedforward networks for image classification with data-efficient training](https://mlsanity.com/paper/2105.03404/) (2021)
- [TSMixer: An All-MLP Architecture for Time Series Forecasting](https://mlsanity.com/paper/2303.06053/) (2023)
- [Unsupervised Representation Learning for Time Series with Temporal Neighborhood Coding](https://mlsanity.com/paper/2106.00750/) (2021)
- [Less Is More: Fast Multivariate Time Series Forecasting with Light Sampling-oriented MLP Structures](https://mlsanity.com/paper/2207.01186/) (2022)
- [CycleMLP: A MLP-like Architecture for Dense Prediction](https://mlsanity.com/paper/2107.10224/) (2021)
- [Unsupervised Time-Series Representation Learning with Iterative Bilinear Temporal-Spectral Fusion](https://mlsanity.com/paper/2202.04770/) (2022)
- [S2-MLP: Spatial-Shift MLP Architecture for Vision](https://mlsanity.com/paper/2106.07477/) (2021)
- [Hire-MLP: Vision MLP via Hierarchical Rearrangement](https://mlsanity.com/paper/2108.13341/) (2021)
- [An Image Patch is a Wave: Phase-Aware Vision MLP](https://mlsanity.com/paper/2111.12294/) (2021)
- [MLP4Rec: A Pure MLP Architecture for Sequential Recommendations](https://mlsanity.com/paper/2204.11510/) (2022)
- [Hierarchical Proxy Modeling for Improved HPO in Time Series Forecasting](https://mlsanity.com/paper/2211.15092/) (2022)

## Built on by (most cited first, 30 of 139)

- [iTransformer: Inverted Transformers Are Effective for Time Series Forecasting](https://mlsanity.com/paper/2310.06625/) (2023)
- [TEMPO: Prompt-based Generative Pre-trained Transformer for Time Series Forecasting](https://mlsanity.com/paper/2310.04948/) (2023)
- [Tiny Time Mixers (TTMs): Fast Pre-trained Models for Enhanced Zero/Few-Shot Forecasting of Multivariate Time Series](https://mlsanity.com/paper/2401.03955/) (2024)
- [TimeMixer++: A General Time Series Pattern Machine for Universal Predictive Analysis](https://mlsanity.com/paper/2410.16032/) (2024)
- [Large Models for Time Series and Spatio-Temporal Data: A Survey and Outlook](https://mlsanity.com/paper/2310.10196/) (2023)
- [SOFTS: Efficient Multivariate Time Series Forecasting with Series-Core Fusion](https://mlsanity.com/paper/2404.14197/) (2024)
- [A Comprehensive Survey of Deep Learning for Multivariate Time Series Forecasting: A Channel Strategy Perspective](https://mlsanity.com/paper/2502.10721/) (2025)
- [SegRNN: Segment Recurrent Neural Network for Long-Term Time Series Forecasting](https://mlsanity.com/paper/2308.11200/) (2023)
- [Frequency Adaptive Normalization For Non-stationary Time Series Forecasting](https://mlsanity.com/paper/2409.20371/) (2024)
- [Multi-resolution Time-Series Transformer for Long-term Forecasting](https://mlsanity.com/paper/2311.04147/) (2023)
- [Time Series Modeling for Heart Rate Prediction: From ARIMA to Transformers](https://mlsanity.com/paper/2406.12199/) (2024)
- [Rethinking Channel Dependence for Multivariate Time Series Forecasting: Learning from Leading Indicators](https://mlsanity.com/paper/2401.17548/) (2024)
- [TS-Reasoner: Domain-Oriented Time Series Inference Agents for Reasoning and Automated Analysis](https://mlsanity.com/paper/2410.04047/) (2024)
- [Learning Pattern-Specific Experts for Time Series Forecasting Under Patch-level Distribution Shift](https://mlsanity.com/paper/2410.09836/) (2024)
- [Towards Urban General Intelligence: A Review and Outlook of Urban Foundation Models](https://mlsanity.com/paper/2402.01749/) (2024)
- [RWKV-TS: Beyond Traditional Recurrent Neural Network for Time Series Tasks](https://mlsanity.com/paper/2401.09093/) (2024)
- [CrossLinear: Plug-and-Play Cross-Correlation Embedding for Time Series Forecasting with Exogenous Variables](https://mlsanity.com/paper/2505.23116/) (2025)
- [MIRA: Medical Time Series Foundation Model for Real-World Health Data](https://mlsanity.com/paper/2506.07584/) (2025)
- [Ada-MSHyper: Adaptive Multi-Scale Hypergraph Transformer for Time Series Forecasting](https://mlsanity.com/paper/2410.23992/) (2024)
- [TimeRAG: BOOSTING LLM Time Series Forecasting via Retrieval-Augmented Generation](https://mlsanity.com/paper/2412.16643/) (2024)
- [True Zero-Shot Inference of Dynamical Systems Preserving Long-Term Statistics](https://mlsanity.com/paper/2505.13192/) (2025)
- [Are Self-Attentions Effective for Time Series Forecasting?](https://mlsanity.com/paper/2405.16877/) (2024)
- [Out-of-Distribution Generalization in Time Series: A Survey](https://mlsanity.com/paper/2503.13868/) (2025)
- [MambaTS: Improved Selective State Space Models for Long-term Time Series Forecasting](https://mlsanity.com/paper/2405.16440/) (2024)
- [Parametric Augmentation for Time Series Contrastive Learning](https://mlsanity.com/paper/2402.10434/) (2024)
- [TSPulse: Tiny Pre-Trained Models with Disentangled Representations for Rapid Time-Series Analysis](https://mlsanity.com/paper/2505.13033/) (2025)
- [Deep Learning for Time Series Forecasting: A Survey](https://mlsanity.com/paper/2503.10198/) (2025)
- [Continual Learning for Smart City: A Survey](https://mlsanity.com/paper/2404.00983/) (2024)
- [A Mamba Foundation Model for Time Series Forecasting](https://mlsanity.com/paper/2411.02941/) (2024)
- [EEO-TFV: Escape-Explore Optimizer for Web-Scale Time-Series Forecasting and Vision Analysis](https://mlsanity.com/paper/2602.02551/) (2026)

## Abstract (arXiv)

Transformers have gained popularity in time series forecasting for their ability to capture long-sequence interactions. However, their high memory and computing requirements pose a critical bottleneck for long-term forecasting. To address this, we propose TSMixer, a lightweight neural architecture exclusively composed of multi-layer perceptron (MLP) modules for multivariate forecasting and representation learning on patched time series. Inspired by MLP-Mixer's success in computer vision, we adapt it for time series, addressing challenges and introducing validated components for enhanced accuracy. This includes a novel design paradigm of attaching online reconciliation heads to the MLP-Mixer backbone, for explicitly modeling the time-series properties such as hierarchy and channel-correlations. We also propose a novel Hybrid channel modeling and infusion of a simple gating approach to effectively handle noisy channel interactions and generalization across diverse datasets. By incorporating these lightweight components, we significantly enhance the learning capability of simple MLP structures, outperforming complex Transformer models with minimal computing usage. Moreover, TSMixer's modular design enables compatibility with both supervised and masked self-supervised learning methods, making it a promising building block for time-series Foundation Models. TSMixer outperforms state-of-the-art MLP and Transformer models in forecasting by a considerable margin of 8-60%. It also outperforms the latest strong benchmarks of Patch-Transformer models (by 1-2%) with a significant reduction in memory and runtime (2-3X). The source code of our model is officially released as PatchTSMixer in the HuggingFace. Model: https://huggingface.co/docs/transformers/main/en/model_doc/patchtsmixer Examples: https://github.com/ibm/tsfm/#notebooks-links