# Attention module improves both performance and interpretability of 4D fMRI decoding neural network

arXiv: https://arxiv.org/abs/2110.00920 · HTML: https://mlsanity.com/paper/2110.00920/

## Key facts

- Cited by 2 papers in the map (data through 14 Oct 2026).
- Builds on 16 of its 40 references that are in the map.
- First posted on arXiv 3 Oct 2021.

## Builds on (most cited first, 16 of 16)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [PyTorch: An Imperative Style, High-Performance Deep Learning Library](https://mlsanity.com/paper/1912.01703/) (2019)
- [U-Net: Convolutional Networks for Biomedical Image Segmentation](https://mlsanity.com/paper/1505.04597/) (2015)
- [CBAM: Convolutional Block Attention Module](https://mlsanity.com/paper/1807.06521/) (2018)
- [Deep learning for time series classification: a review](https://mlsanity.com/paper/1809.04356/) (2018)
- [Residual Attention Network for Image Classification](https://mlsanity.com/paper/1704.06904/) (2017)
- [Convolutional Neural Networks as a Model of the Visual System: Past, Present, and Future](https://mlsanity.com/paper/2001.07092/) (2020)
- [BOLD5000: A public fMRI dataset of 5000 images](https://mlsanity.com/paper/1809.01281/) (2018)
- [The Dynamics of Functional Brain Networks: Integrated Network States during Cognitive Function](https://mlsanity.com/paper/1511.02976/) (2015)
- [Attend and Decode: 4D fMRI Task State Decoding Using Attention Models](https://mlsanity.com/paper/2004.05234/) (2020)
- [Decoding and mapping task states of the human brain via deep learning](https://mlsanity.com/paper/1801.09858/) (2018)
- [Deep Transfer Learning For Whole-Brain fMRI Analyses](https://mlsanity.com/paper/1907.01953/) (2019)
- [Brain decoding from functional MRI using long short-term memory recurrent neural networks](https://mlsanity.com/paper/1809.05561/) (2018)
- [Deep learning with 4D spatio-temporal data representations for OCT-based force estimation](https://mlsanity.com/paper/2005.10033/) (2020)
- [Convolutional Neural Network Interpretability with General Pattern Theory](https://mlsanity.com/paper/2102.04247/) (2021)

## Built on by (most cited first, 2 of 2)

- [Domain Adaptation and Generalization on Functional Medical Images: A Systematic Survey](https://mlsanity.com/paper/2212.03176/) (2022)
- [Hi-DREAM: Brain-Inspired Hierarchical Diffusion for fMRI-to-Image Reconstruction via ROI Encoder and VisuAl Mapping](https://mlsanity.com/paper/2511.11437/) (2025)

## Abstract (arXiv)

Decoding brain cognitive states from neuroimaging signals is an important topic in neuroscience. In recent years, deep neural networks (DNNs) have been recruited for multiple brain state decoding and achieved good performance. However, the open question of how to interpret the DNN black box remains unanswered. Capitalizing on advances in machine learning, we integrated attention modules into brain decoders to facilitate an in-depth interpretation of DNN channels. A 4D convolution operation was also included to extract temporo-spatial interaction within the fMRI signal. The experiments showed that the proposed model obtains a very high accuracy (97.4%) and outperforms previous researches on the 7 different task benchmarks from the Human Connectome Project (HCP) dataset. The visualization analysis further illustrated the hierarchical emergence of task-specific masks with depth. Finally, the model was retrained to regress individual traits within the HCP and to classify viewing images from the BOLD5000 dataset, respectively. Transfer learning also achieves good performance. A further visualization analysis shows that, after transfer learning, low-level attention masks remained similar to the source domain, whereas high-level attention masks changed adaptively. In conclusion, the proposed 4D model with attention module performed well and facilitated interpretation of DNNs, which is helpful for subsequent research.