# BolT: Fused Window Transformers for fMRI Time Series Analysis

arXiv: https://arxiv.org/abs/2205.11578 · HTML: https://mlsanity.com/paper/2205.11578/

## Key facts

- Cited by 23 papers in the map (data through 14 Oct 2026).
- Builds on 33 of its 152 references that are in the map.
- First posted on arXiv 23 May 2022.

## Builds on (most cited first, 30 of 33)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [Language Models are Few-Shot Learners](https://mlsanity.com/paper/2005.14165/) (2020)
- [An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale](https://mlsanity.com/paper/2010.11929/) (2020)
- [Swin Transformer: Hierarchical Vision Transformer using Shifted Windows](https://mlsanity.com/paper/2103.14030/) (2021)
- [Masked Autoencoders Are Scalable Vision Learners](https://mlsanity.com/paper/2111.06377/) (2021)
- [Longformer: The Long-Document Transformer](https://mlsanity.com/paper/2004.05150/) (2020)
- [On the Bottleneck of Graph Neural Networks and its Practical Implications](https://mlsanity.com/paper/2006.05205/) (2020)
- [Measuring and Relieving the Over-smoothing Problem for Graph Neural Networks from the Topological View](https://mlsanity.com/paper/1909.03211/) (2019)
- [Generic Attention-model Explainability for Interpreting Bi-Modal and Encoder-Decoder Transformers](https://mlsanity.com/paper/2103.15679/) (2021)
- [Focal Self-attention for Local-Global Interactions in Vision Transformers](https://mlsanity.com/paper/2107.00641/) (2021)
- [ResViT: Residual vision transformers for multi-modal medical image synthesis](https://mlsanity.com/paper/2106.16031/) (2021)
- [Disease Prediction using Graph Convolutional Networks: Application to Autism Spectrum Disorder and Alzheimer's Disease](https://mlsanity.com/paper/1806.01738/) (2018)
- [Learning Dynamic Graph Representation of Brain Connectome with Spatio-Temporal Attention](https://mlsanity.com/paper/2105.13495/) (2021)
- [Unsupervised MRI Reconstruction via Zero-Shot Learned Adversarial Transformers](https://mlsanity.com/paper/2105.08059/) (2021)
- [Spatio-Temporal Graph Convolution for Resting-State fMRI Analysis](https://mlsanity.com/paper/2003.10613/) (2020)
- [Reconstructing Faces from fMRI Patterns using Deep Generative Neural Networks](https://mlsanity.com/paper/1810.03856/) (2018)
- [Spectral Graph Convolutions for Population-based Disease Prediction](https://mlsanity.com/paper/1703.03020/) (2017)
- [Understanding Graph Isomorphism Network for rs-fMRI Functional Connectivity Analysis](https://mlsanity.com/paper/2001.03690/) (2020)
- [Graph Neural Network for Interpreting Task-fMRI Biomarkers](https://mlsanity.com/paper/1907.01661/) (2019)
- [Federated Learning of Generative Image Priors for MRI Reconstruction](https://mlsanity.com/paper/2202.04175/) (2022)
- [Deriving reproducible biomarkers from multi-site resting-state data: An Autism-based example](https://mlsanity.com/paper/1611.06066/) (2016)
- [Graph Saliency Maps through Spectral Convolutional Networks: Application to Sex Classification with Brain Connectivity](https://mlsanity.com/paper/1806.01764/) (2018)
- [Input-Cell Attention Reduces Vanishing Saliency of Recurrent Neural Networks](https://mlsanity.com/paper/1910.12370/) (2019)
- [Deep Learning with Mixed Supervision for Brain Tumor Segmentation](https://mlsanity.com/paper/1812.04571/) (2018)
- [TranSMS: Transformers for Super-Resolution Calibration in Magnetic Particle Imaging](https://mlsanity.com/paper/2111.02163/) (2021)
- [Attend and Decode: 4D fMRI Task State Decoding Using Attention Models](https://mlsanity.com/paper/2004.05234/) (2020)
- [Resting state fMRI functional connectivity-based classification using a convolutional neural network architecture](https://mlsanity.com/paper/1707.06682/) (2017)
- [Classification of Alzheimer's Disease using fMRI Data and Deep Learning Convolutional Neural Networks](https://mlsanity.com/paper/1603.08631/) (2016)
- [Deep learning of fMRI big data: a novel approach to subject-transfer decoding](https://mlsanity.com/paper/1502.00093/) (2015)

## Built on by (most cited first, 23 of 23)

- [SwiFT: Swin 4D fMRI Transformer](https://mlsanity.com/paper/2307.05916/) (2023)
- [Transforming medical imaging with Transformers? A comparative review of key properties, current progresses, and future perspectives](https://mlsanity.com/paper/2206.01136/) (2022)
- [I2I-Mamba: Multi-modal medical image synthesis via selective state space modeling](https://mlsanity.com/paper/2405.14022/) (2024)
- [DreaMR: Diffusion-driven Counterfactual Explanation for Functional MRI](https://mlsanity.com/paper/2307.09547/) (2023)
- [fMRI-LM: Towards a Universal Foundation Model for Language-Aligned fMRI Understanding](https://mlsanity.com/paper/2511.21760/) (2025)
- [HydraViT: Adaptive Multi-Branch Transformer for Multi-Label Disease Classification from Chest X-ray Images](https://mlsanity.com/paper/2310.06143/) (2023)
- [NeuroPath: A Neural Pathway Transformer for Joining the Dots of Human Connectomes](https://mlsanity.com/paper/2409.17510/) (2024)
- [Brain-Semantoks: Learning Semantic Tokens of Brain Dynamics with a Self-Distilled Foundation Model](https://mlsanity.com/paper/2512.11582/) (2025)
- [MCI Detection using fMRI time series embeddings of Recurrence plots](https://mlsanity.com/paper/2311.18265/) (2023)
- [NeuroMambaLLM: Dynamic Graph Learning of fMRI Functional Connectivity in Autistic Brains Using Mamba and Language Model Reasoning](https://mlsanity.com/paper/2602.13770/) (2026)
- [Machine Learning on Dynamic Functional Connectivity: Promise, Pitfalls, and Interpretations](https://mlsanity.com/paper/2409.11377/) (2024)
- [STARFormer: A Novel Spatio-Temporal Aggregation Reorganization Transformer of FMRI for Brain Disorder Diagnosis](https://mlsanity.com/paper/2501.00378/) (2024)
- [Hierarchical Characterization of Brain Dynamics via State Space-based Vector Quantization](https://mlsanity.com/paper/2506.22952/) (2025)
- [Resting-state fMRI Analysis using Quantum Time-series Transformer](https://mlsanity.com/paper/2509.00711/) (2025)
- [VarCoNet: A variability-aware self-supervised framework for functional connectome extraction from resting-state fMRI](https://mlsanity.com/paper/2510.02120/) (2025)
- [Self-Supervised Pretraining on Paired Sequences of fMRI Data for Transfer Learning to Brain Decoding Tasks](https://mlsanity.com/paper/2305.09057/) (2023)
- [Sequential Transfer Learning to Decode Heard and Imagined Timbre from fMRI Data](https://mlsanity.com/paper/2305.13226/) (2023)
- [Reconstructing physiological signals from fMRI across the adult lifespan](https://mlsanity.com/paper/2408.14453/) (2024)
- [Multi-Task Adversarial Variational Autoencoder for Estimating Biological Brain Age with Multimodal Neuroimaging](https://mlsanity.com/paper/2411.10100/) (2024)
- [Deep Learning Inductive Biases for fMRI Time Series Classification during Resting-state and Movie-watching](https://mlsanity.com/paper/2509.16973/) (2025)
- [NeuroSSM: Multiscale Differential State-Space Modeling for Context-Aware fMRI Analysis](https://mlsanity.com/paper/2601.01229/) (2026)
- [Nonlocal operator learning for fMRI encoding and decoding tasks](https://mlsanity.com/paper/2605.20389/) (2026)
- [UniAR: A Unified Framework for Autism Recognition Enhanced by Multi-View Prompt Learning](https://mlsanity.com/paper/2609.31298/) (2026)

## Abstract (arXiv)

Deep-learning models have enabled performance leaps in analysis of high-dimensional functional MRI (fMRI) data. Yet, many previous methods are suboptimally sensitive for contextual representations across diverse time scales. Here, we present BolT, a blood-oxygen-level-dependent transformer model, for analyzing multi-variate fMRI time series. BolT leverages a cascade of transformer encoders equipped with a novel fused window attention mechanism. Encoding is performed on temporally-overlapped windows within the time series to capture local representations. To integrate information temporally, cross-window attention is computed between base tokens in each window and fringe tokens from neighboring windows. To gradually transition from local to global representations, the extent of window overlap and thereby number of fringe tokens are progressively increased across the cascade. Finally, a novel cross-window regularization is employed to align high-level classification features across the time series. Comprehensive experiments on large-scale public datasets demonstrate the superior performance of BolT against state-of-the-art methods. Furthermore, explanatory analyses to identify landmark time points and regions that contribute most significantly to model decisions corroborate prominent neuroscientific findings in the literature.