# Hierarchical Spatio-Temporal State-Space Modeling for fMRI Analysis

arXiv: https://arxiv.org/abs/2408.13074 · HTML: https://mlsanity.com/paper/2408.13074/

## Key facts

- Cited by 6 papers in the map (data through 14 Oct 2026).
- Placed in Physiological signal modeling, one of 10 directions in State-space sequence models (S4, Mamba).
- Builds on 18 of its 48 references that are in the map.
- First posted on arXiv 23 Aug 2024.

## Placed in

- State-space sequence models (S4, Mamba) / [Physiological signal modeling](https://mlsanity.com/ssm/physiological-signal-modeling/)

## Builds on (most cited first, 18 of 18)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [A Unified Approach to Interpreting Model Predictions](https://mlsanity.com/paper/1705.07874/) (2017)
- [Mamba: Linear-Time Sequence Modeling with Selective State Spaces](https://mlsanity.com/paper/2312.00752/) (2023)
- [RoFormer: Enhanced Transformer with Rotary Position Embedding](https://mlsanity.com/paper/2104.09864/) (2021)
- [A Comprehensive Survey on Graph Neural Networks](https://mlsanity.com/paper/1901.00596/) (2019)
- [Vision Mamba: Efficient Visual Representation Learning with Bidirectional State Space Model](https://mlsanity.com/paper/2401.09417/) (2024)
- [HiPPO: Recurrent Memory with Optimal Polynomial Projections](https://mlsanity.com/paper/2008.07669/) (2020)
- [Captum: A unified and generic model interpretability library for PyTorch](https://mlsanity.com/paper/2009.07896/) (2020)
- [VideoMamba: State Space Model for Efficient Video Understanding](https://mlsanity.com/paper/2403.06977/) (2024)
- [Graph-Mamba: Towards Long-Range Graph Sequence Modeling with Selective State Spaces](https://mlsanity.com/paper/2402.00789/) (2024)
- [Mamba-UNet: UNet-Like Pure Visual Mamba for Medical Image Segmentation](https://mlsanity.com/paper/2402.05079/) (2024)
- [FBNETGEN: Task-aware GNN-based fMRI Analysis via Functional Brain Network Generation](https://mlsanity.com/paper/2205.12465/) (2022)
- [SwiFT: Swin 4D fMRI Transformer](https://mlsanity.com/paper/2307.05916/) (2023)
- [FT2TF: First-Person Statement Text-To-Talking Face Generation](https://mlsanity.com/paper/2312.05430/) (2023)
- [DenseMamba: State Space Models with Dense Hidden Connection for Efficient Large Language Models](https://mlsanity.com/paper/2403.00818/) (2024)
- [Deform-Mamba Network for MRI Super-Resolution](https://mlsanity.com/paper/2407.05969/) (2024)
- [A Deep Network for Explainable Prediction of Non-Imaging Phenotypes using Anatomical Multi-View Data](https://mlsanity.com/paper/2401.04579/) (2024)
- [Age Prediction Performance Varies Across Deep, Superficial, and Cerebellar White Matter Connections](https://mlsanity.com/paper/2211.07398/) (2022)

## Built on by (most cited first, 6 of 6)

- [MoRE-Brain: Routed Mixture of Experts for Interpretable and Generalizable Cross-Subject fMRI Visual Decoding](https://mlsanity.com/paper/2505.15946/) (2025)
- [Learning Sparsity for Effective and Efficient Music Performance Question Answering](https://mlsanity.com/paper/2506.01319/) (2025)
- [4D Multimodal Co-attention Fusion Network with Latent Contrastive Alignment for Alzheimer's Diagnosis](https://mlsanity.com/paper/2504.16798/) (2025)
- [BrainATCL: Adaptive Temporal Brain Connectivity Learning for Functional Link Prediction and Age Estimation](https://mlsanity.com/paper/2508.07106/) (2025)
- [Foundation-Model-Boosted Multimodal Learning for fMRI-based Neuropathic Pain Drug Response Prediction](https://mlsanity.com/paper/2503.00210/) (2025)
- [NeuroSSM: Multiscale Differential State-Space Modeling for Context-Aware fMRI Analysis](https://mlsanity.com/paper/2601.01229/) (2026)

## Abstract (arXiv)

Recent advances in deep learning structured state space models, especially the Mamba architecture, have demonstrated remarkable performance improvements while maintaining linear complexity. In this study, we introduce functional spatiotemporal Mamba (FST-Mamba), a Mamba-based model designed for discovering neurological biomarkers using functional magnetic resonance imaging (fMRI). We focus on dynamic functional network connectivity (dFNC) derived from fMRI and propose a hierarchical spatiotemporal Mamba-based network that processes spatial and temporal information separately using Mamba-based encoders. Leveraging the topological uniqueness of the FNC matrix, we introduce a component-wise varied-scale aggregation (CVA) mechanism to aggregate connectivity across individual components within brain networks, enabling the model to capture component-level and network-level information. Additionally, we propose symmetric rotary position encoding (SymRope) to encode the relative positions of each functional connection while considering the symmetric nature of the FNC matrix. Experimental results demonstrate significant improvements in the proposed FST-Mamba model on various brain-based classification and regression tasks. We further show brain connectivities and dynamics that are crucial for the prediction. Our work reveals the substantial potential of attention-free sequence modeling in brain discovery. The codes are publicly available here: https://github.com/yuxiangwei0808/FunctionalMamba/tree/main.