# Disentangling Spatial-Temporal Functional Brain Networks via Twin-Transformers

arXiv: https://arxiv.org/abs/2204.09225 · HTML: https://mlsanity.com/paper/2204.09225/

## Key facts

- Cited by 6 papers in the map (data through 14 Oct 2026).
- Builds on 6 of its 24 references that are in the map.
- First posted on arXiv 20 Apr 2022.

## Builds on (most cited first, 6 of 6)

- [An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale](https://mlsanity.com/paper/2010.11929/) (2020)
- [Swin Transformer: Hierarchical Vision Transformer using Shifted Windows](https://mlsanity.com/paper/2103.14030/) (2021)
- [Masked Autoencoders Are Scalable Vision Learners](https://mlsanity.com/paper/2111.06377/) (2021)
- [ViViT: A Video Vision Transformer](https://mlsanity.com/paper/2103.15691/) (2021)
- [Frozen in Time: A Joint Video and Image Encoder for End-to-End Retrieval](https://mlsanity.com/paper/2104.00650/) (2021)
- [Attend and Decode: 4D fMRI Task State Decoding Using Attention Models](https://mlsanity.com/paper/2004.05234/) (2020)

## Built on by (most cited first, 6 of 6)

- [Review of Large Vision Models and Visual Prompt Engineering](https://mlsanity.com/paper/2307.00855/) (2023)
- [SAMAug: Point Prompt Augmentation for Segment Anything Model](https://mlsanity.com/paper/2307.01187/) (2023)
- [Transforming medical imaging with Transformers? A comparative review of key properties, current progresses, and future perspectives](https://mlsanity.com/paper/2206.01136/) (2022)
- [BolT: Fused Window Transformers for fMRI Time Series Analysis](https://mlsanity.com/paper/2205.11578/) (2022)
- [DreaMR: Diffusion-driven Counterfactual Explanation for Functional MRI](https://mlsanity.com/paper/2307.09547/) (2023)
- [DCMM-Transformer: Degree-Corrected Mixed-Membership Attention for Medical Imaging](https://mlsanity.com/paper/2511.12047/) (2025)

## Abstract (arXiv)

How to identify and characterize functional brain networks (BN) is fundamental to gain system-level insights into the mechanisms of brain organizational architecture. Current functional magnetic resonance (fMRI) analysis highly relies on prior knowledge of specific patterns in either spatial (e.g., resting-state network) or temporal (e.g., task stimulus) domain. In addition, most approaches aim to find group-wise common functional networks, individual-specific functional networks have been rarely studied. In this work, we propose a novel Twin-Transformers framework to simultaneously infer common and individual functional networks in both spatial and temporal space, in a self-supervised manner. The first transformer takes space-divided information as input and generates spatial features, while the second transformer takes time-related information as input and outputs temporal features. The spatial and temporal features are further separated into common and individual ones via interactions (weights sharing) and constraints between the two transformers. We applied our TwinTransformers to Human Connectome Project (HCP) motor task-fMRI dataset and identified multiple common brain networks, including both task-related and resting-state networks (e.g., default mode network). Interestingly, we also successfully recovered a set of individual-specific networks that are not related to task stimulus and only exist at the individual level.