# Self-Clustering Graph Transformer Approach to Model Resting-State Functional Brain Activity

arXiv: https://arxiv.org/abs/2501.16345 · HTML: https://mlsanity.com/paper/2501.16345/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Builds on 6 of its 21 references that are in the map.
- First posted on arXiv 17 Jan 2025.

## Builds on (most cited first, 6 of 6)

- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [PyTorch: An Imperative Style, High-Performance Deep Learning Library](https://mlsanity.com/paper/1912.01703/) (2019)
- [Fast Graph Representation Learning with PyTorch Geometric](https://mlsanity.com/paper/1903.02428/) (2019)
- [A Generalization of Transformer Networks to Graphs](https://mlsanity.com/paper/2012.09699/) (2020)
- [Brain Networks and Intelligence: A Graph Neural Network Based Approach to Resting State fMRI Data](https://mlsanity.com/paper/2311.03520/) (2023)
- [DSAM: A Deep Learning Framework for Analyzing Temporal and Spatial Dynamics in Brain Networks](https://mlsanity.com/paper/2405.15805/) (2024)

## Built on by (most cited first, 1 of 1)

- [Fast Clustering of Categorical Big Data](https://mlsanity.com/paper/2502.07081/) (2025)

## Abstract (arXiv)

Resting-state functional magnetic resonance imaging (rs-fMRI) offers valuable insights into the human brain's functional organization and is a powerful tool for investigating the relationship between brain function and cognitive processes, as it allows for the functional organization of the brain to be captured without relying on a specific task or stimuli. In this study, we introduce a novel attention mechanism for graphs with subnetworks, named Self-Clustering Graph Transformer (SCGT), designed to handle the issue of uniform node updates in graph transformers. By using static functional connectivity (FC) correlation features as input to the transformer model, SCGT effectively captures the sub-network structure of the brain by performing cluster-specific updates to the nodes, unlike uniform node updates in vanilla graph transformers, further allowing us to learn and interpret the subclusters. We validate our approach on the Adolescent Brain Cognitive Development (ABCD) dataset, comprising 7,957 participants, for the prediction of total cognitive score and gender classification. Our results demonstrate that SCGT outperforms the vanilla graph transformer method and other recent models, offering a promising tool for modeling brain functional connectivity and interpreting the underlying subnetwork structures.