# Large-scale Graph Representation Learning of Dynamic Brain Connectome with Transformers

arXiv: https://arxiv.org/abs/2312.14939 · HTML: https://mlsanity.com/paper/2312.14939/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Builds on 12 of its 26 references that are in the map.
- First posted on arXiv 4 Dec 2023.

## Builds on (most cited first, 12 of 12)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [A Generalization of Transformer Networks to Graphs](https://mlsanity.com/paper/2012.09699/) (2020)
- [A Survey on Oversmoothing in Graph Neural Networks](https://mlsanity.com/paper/2303.10993/) (2023)
- [Transformer for Graphs: An Overview from Architecture Perspective](https://mlsanity.com/paper/2202.08455/) (2022)
- [Brain Network Transformer](https://mlsanity.com/paper/2210.06681/) (2022)
- [Graph Neural Networks in Network Neuroscience](https://mlsanity.com/paper/2106.03535/) (2021)
- [Learning Dynamic Graph Representation of Brain Connectome with Spatio-Temporal Attention](https://mlsanity.com/paper/2105.13495/) (2021)
- [Attending to Graph Transformers](https://mlsanity.com/paper/2302.04181/) (2023)
- [Understanding Graph Isomorphism Network for rs-fMRI Functional Connectivity Analysis](https://mlsanity.com/paper/2001.03690/) (2020)
- [Anomaly Detection in Multiplex Dynamic Networks: from Blockchain Security to Brain Disease Prediction](https://mlsanity.com/paper/2211.08378/) (2022)
- [Beyond the Snapshot: Brain Tokenized Graph Transformer for Longitudinal Brain Functional Connectome Embedding](https://mlsanity.com/paper/2307.00858/) (2023)
- [DynDepNet: Learning Time-Varying Dependency Structures from fMRI Data via Dynamic Graph Structure Learning](https://mlsanity.com/paper/2209.13513/) (2022)

## Built on by (most cited first, 1 of 1)

- [Towards a general-purpose foundation model for fMRI analysis](https://mlsanity.com/paper/2506.11167/) (2025)

## Abstract (arXiv)

Graph Transformers have recently been successful in various graph representation learning tasks, providing a number of advantages over message-passing Graph Neural Networks. Utilizing Graph Transformers for learning the representation of the brain functional connectivity network is also gaining interest. However, studies to date have underlooked the temporal dynamics of functional connectivity, which fluctuates over time. Here, we propose a method for learning the representation of dynamic functional connectivity with Graph Transformers. Specifically, we define the connectome embedding, which holds the position, structure, and time information of the functional connectivity graph, and use Transformers to learn its representation across time. We perform experiments with over 50,000 resting-state fMRI samples obtained from three datasets, which is the largest number of fMRI data used in studies by far. The experimental results show that our proposed method outperforms other competitive baselines in gender classification and age regression tasks based on the functional connectivity extracted from the fMRI data.