# Attend and Decode: 4D fMRI Task State Decoding Using Attention Models

arXiv: https://arxiv.org/abs/2004.05234 · HTML: https://mlsanity.com/paper/2004.05234/

## Key facts

- Cited by 17 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 10 Apr 2020.

## Built on by (most cited first, 17 of 17)

- [Learning Dynamic Graph Representation of Brain Connectome with Spatio-Temporal Attention](https://mlsanity.com/paper/2105.13495/) (2021)
- [SwiFT: Swin 4D fMRI Transformer](https://mlsanity.com/paper/2307.05916/) (2023)
- [BolT: Fused Window Transformers for fMRI Time Series Analysis](https://mlsanity.com/paper/2205.11578/) (2022)
- [Towards a general-purpose foundation model for fMRI analysis](https://mlsanity.com/paper/2506.11167/) (2025)
- [SLIM-Brain: A Data- and Training-Efficient Foundation Model for fMRI Data Analysis](https://mlsanity.com/paper/2512.21881/) (2025)
- [Disentangling Spatial-Temporal Functional Brain Networks via Twin-Transformers](https://mlsanity.com/paper/2204.09225/) (2022)
- [See Through Their Minds: Learning Transferable Neural Representation from Cross-Subject fMRI](https://mlsanity.com/paper/2403.06361/) (2024)
- [Challenges for cognitive decoding using deep learning methods](https://mlsanity.com/paper/2108.06896/) (2021)
- [Evaluating deep transfer learning for whole-brain cognitive decoding](https://mlsanity.com/paper/2111.01562/) (2021)
- [Attention module improves both performance and interpretability of 4D fMRI decoding neural network](https://mlsanity.com/paper/2110.00920/) (2021)
- [Looking deeper into interpretable deep learning in neuroimaging: a comprehensive survey](https://mlsanity.com/paper/2307.09615/) (2023)
- [BrainMT: A Hybrid Mamba-Transformer Architecture for Modeling Long-Range Dependencies in Functional MRI Data](https://mlsanity.com/paper/2506.22591/) (2025)
- [VarCoNet: A variability-aware self-supervised framework for functional connectome extraction from resting-state fMRI](https://mlsanity.com/paper/2510.02120/) (2025)
- [Self-Supervised Pretraining on Paired Sequences of fMRI Data for Transfer Learning to Brain Decoding Tasks](https://mlsanity.com/paper/2305.09057/) (2023)
- [Sequential Transfer Learning to Decode Heard and Imagined Timbre from fMRI Data](https://mlsanity.com/paper/2305.13226/) (2023)
- [Brain Effective Connectivity Estimation via Fourier Spatiotemporal Attention](https://mlsanity.com/paper/2503.11283/) (2025)
- [NeuroSSM: Multiscale Differential State-Space Modeling for Context-Aware fMRI Analysis](https://mlsanity.com/paper/2601.01229/) (2026)

## Abstract (arXiv)

Functional magnetic resonance imaging (fMRI) is a neuroimaging modality that captures the blood oxygen level in a subject's brain while the subject either rests or performs a variety of functional tasks under different conditions. Given fMRI data, the problem of inferring the task, known as task state decoding, is challenging due to the high dimensionality (hundreds of million sampling points per datum) and complex spatio-temporal blood flow patterns inherent in the data. In this work, we propose to tackle the fMRI task state decoding problem by casting it as a 4D spatio-temporal classification problem. We present a novel architecture called Brain Attend and Decode (BAnD), that uses residual convolutional neural networks for spatial feature extraction and self-attention mechanisms for temporal modeling. We achieve significant performance gain compared to previous works on a 7-task benchmark from the large-scale Human Connectome Project-Young Adult (HCP-YA) dataset. We also investigate the transferability of BAnD's extracted features on unseen HCP tasks, either by freezing the spatial feature extraction layers and retraining the temporal model, or finetuning the entire model. The pre-trained features from BAnD are useful on similar tasks while finetuning them yields competitive results on unseen tasks/conditions.