# FMamba: Mamba based on Fast-attention for Multivariate Time-series Forecasting

arXiv: https://arxiv.org/abs/2407.14814 · HTML: https://mlsanity.com/paper/2407.14814/

## Key facts

- Cited by 8 papers in the map (data through 14 Oct 2026).
- Placed in Time series forecasting with SSMs, one of 10 directions in State-space sequence models (S4, Mamba).
- Builds on 7 of its 15 references that are in the map.
- First posted on arXiv 20 Jul 2024.

## Placed in

- State-space sequence models (S4, Mamba) / [Time series forecasting with SSMs](https://mlsanity.com/ssm/time-series-forecasting-with-ssms/)

## Builds on (most cited first, 7 of 7)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [Mamba: Linear-Time Sequence Modeling with Selective State Spaces](https://mlsanity.com/paper/2312.00752/) (2023)
- [Efficiently Modeling Long Sequences with Structured State Spaces](https://mlsanity.com/paper/2111.00396/) (2021)
- [Informer: Beyond Efficient Transformer for Long Sequence Time-Series Forecasting](https://mlsanity.com/paper/2012.07436/) (2020)
- [Rethinking Attention with Performers](https://mlsanity.com/paper/2009.14794/) (2020)
- [HiPPO: Recurrent Memory with Optimal Polynomial Projections](https://mlsanity.com/paper/2008.07669/) (2020)
- [SCINet: Time Series Modeling and Forecasting with Sample Convolution and Interaction](https://mlsanity.com/paper/2106.09305/) (2021)

## Built on by (most cited first, 8 of 8)

- [Sequential Order-Robust Mamba for Time Series Forecasting](https://mlsanity.com/paper/2410.23356/) (2024)
- [Channel Normalization for Time Series Channel Identification](https://mlsanity.com/paper/2506.00432/) (2025)
- [Automatic selection of the best neural architecture for time series forecasting](https://mlsanity.com/paper/2501.12215/) (2025)
- [EffiCANet: Efficient Time Series Forecasting with Convolutional Attention](https://mlsanity.com/paper/2411.04669/) (2024)
- [MTS-UNMixers: Multivariate Time Series Forecasting via Channel-Time Dual Unmixing](https://mlsanity.com/paper/2411.17770/) (2024)
- [Contrast Similarity-Aware Dual-Pathway Mamba for Multivariate Time Series Node Classification](https://mlsanity.com/paper/2411.12222/) (2024)
- [DC-Mamber: A Dual Channel Prediction Model based on Mamba and Linear Transformer for Multivariate Time Series Forecasting](https://mlsanity.com/paper/2507.04381/) (2025)
- [DSAT-HD: Dual-Stream Adaptive Transformer with Hybrid Decomposition for Multivariate Time Series Forecasting](https://mlsanity.com/paper/2509.24800/) (2025)

## Abstract (arXiv)

In multivariate time-series forecasting (MTSF), extracting the temporal correlations of the input sequences is crucial. While popular Transformer-based predictive models can perform well, their quadratic computational complexity results in inefficiency and high overhead. The recently emerged Mamba, a selective state space model, has shown promising results in many fields due to its strong temporal feature extraction capabilities and linear computational complexity. However, due to the unilateral nature of Mamba, channel-independent predictive models based on Mamba cannot attend to the relationships among all variables in the manner of Transformer-based models. To address this issue, we combine fast-attention with Mamba to introduce a novel framework named FMamba for MTSF. Technically, we first extract the temporal features of the input variables through an embedding layer, then compute the dependencies among input variables via the fast-attention module. Subsequently, we use Mamba to selectively deal with the input features and further extract the temporal dependencies of the variables through the multi-layer perceptron block (MLP-block). Finally, FMamba obtains the predictive results through the projector, a linear layer. Experimental results on eight public datasets demonstrate that FMamba can achieve state-of-the-art performance while maintaining low computational overhead.