# Contrasformer: A Brain Network Contrastive Transformer for Neurodegenerative Condition Identification

arXiv: https://arxiv.org/abs/2409.10944 · HTML: https://mlsanity.com/paper/2409.10944/

## Key facts

- Cited by 13 papers in the map (data through 14 Oct 2026).
- Builds on 27 of its 56 references that are in the map.
- First posted on arXiv 17 Sep 2024.

## Builds on (most cited first, 27 of 27)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [A Simple Framework for Contrastive Learning of Visual Representations](https://mlsanity.com/paper/2002.05709/) (2020)
- [Graph Attention Networks](https://mlsanity.com/paper/1710.10903/) (2017)
- [Inductive Representation Learning on Large Graphs](https://mlsanity.com/paper/1706.02216/) (2017)
- [How Powerful are Graph Neural Networks?](https://mlsanity.com/paper/1810.00826/) (2018)
- [Neural Message Passing for Quantum Chemistry](https://mlsanity.com/paper/1704.01212/) (2017)
- [Graph Contrastive Learning with Augmentations](https://mlsanity.com/paper/2010.13902/) (2020)
- [Recipe for a General, Powerful, Scalable Graph Transformer](https://mlsanity.com/paper/2205.12454/) (2022)
- [Graph Contrastive Learning with Adaptive Augmentation](https://mlsanity.com/paper/2010.14945/) (2020)
- [Rethinking Graph Transformers with Spectral Attention](https://mlsanity.com/paper/2106.03893/) (2021)
- [Residual Gated Graph ConvNets](https://mlsanity.com/paper/1711.07553/) (2017)
- [Graph Neural Networks with Learnable Structural and Positional Representations](https://mlsanity.com/paper/2110.07875/) (2021)
- [Not too little, not too much: a theoretical analysis of graph (over)smoothing](https://mlsanity.com/paper/2205.12156/) (2022)
- [Machine Learning for Neuroimaging with Scikit-Learn](https://mlsanity.com/paper/1412.3919/) (2014)
- [FBNETGEN: Task-aware GNN-based fMRI Analysis via Functional Brain Network Generation](https://mlsanity.com/paper/2205.12465/) (2022)
- [Distance Metric Learning using Graph Convolutional Networks: Application to Functional Brain Networks](https://mlsanity.com/paper/1703.02161/) (2017)
- [Graph Neural Network for Interpreting Task-fMRI Biomarkers](https://mlsanity.com/paper/1907.01661/) (2019)
- [IMF: Interactive Multimodal Fusion Model for Link Prediction](https://mlsanity.com/paper/2303.10816/) (2023)
- [Explainable Classification of Brain Networks via Contrast Subgraphs](https://mlsanity.com/paper/2006.05176/) (2020)
- [Pooling Regularized Graph Neural Network for fMRI Biomarker Analysis](https://mlsanity.com/paper/2007.14589/) (2020)
- [Exploring the reproducibility of functional connectivity alterations in Parkinson's Disease](https://mlsanity.com/paper/1611.04794/) (2016)
- [Data-Driven Network Neuroscience: On Data Collection and Benchmark](https://mlsanity.com/paper/2211.12421/) (2022)
- [Transformer-Based Hierarchical Clustering for Brain Network Analysis](https://mlsanity.com/paper/2305.04142/) (2023)
- [Dynamic Brain Transformer with Multi-level Attention for Functional Brain Network Analysis](https://mlsanity.com/paper/2309.01941/) (2023)
- [Union Subgraph Neural Networks](https://mlsanity.com/paper/2305.15747/) (2023)
- [A Class-Aware Representation Refinement Framework for Graph Classification](https://mlsanity.com/paper/2209.00936/) (2022)

## Built on by (most cited first, 13 of 13)

- [BrainOOD: Out-of-distribution Generalizable Brain Network Analysis](https://mlsanity.com/paper/2502.01688/) (2025)
- [Multi-Atlas Brain Network Classification through Consistency Distillation and Complementary Information Fusion](https://mlsanity.com/paper/2410.08228/) (2024)
- [BrainPrompt: Multi-Level Brain Prompt Enhancement for Neurological Condition Identification](https://mlsanity.com/paper/2504.16096/) (2025)
- [Ada-FCN: Adaptive Frequency-Coupled Network for fMRI-Based Brain Disorder Classification](https://mlsanity.com/paper/2511.04718/) (2025)
- [Moving Beyond Functional Connectivity: Time-Series Modeling for fMRI-Based Brain Disorder Classification](https://mlsanity.com/paper/2602.08262/) (2026)
- [BrainHGT: A Hierarchical Graph Transformer for Interpretable Brain Network Analysis](https://mlsanity.com/paper/2511.17604/) (2025)
- [NOCL: Node-Oriented Conceptualization LLM for Graph Tasks without Message Passing](https://mlsanity.com/paper/2506.10014/) (2025)
- [Toward a Multi-View Brain Network Foundation Model: Cross-View Consistency Learning Across Arbitrary Atlases](https://mlsanity.com/paper/2603.20348/) (2026)
- [DynFS-MoE: Dynamic Functional-Structural Mixture-of-Experts for Post-Traumatic Epilepsy Diagnosis](https://mlsanity.com/paper/2606.16203/) (2026)
- [Artemis: Anatomy-Resolved inTervention for Eliminating Multimodal NeuroImage confounderS](https://mlsanity.com/paper/2606.18287/) (2026)
- [Navigating Hierarchy: Hyperbolic Learning on Brain Graphs for Disorder Diagnosis](https://mlsanity.com/paper/2607.07077/) (2026)
- [MPP-GNN: Subject-Adaptive Community Detection for fMRI-Based Alzheimer's Disease Classification](https://mlsanity.com/paper/2607.28681/) (2026)
- [fMRI-TAMCL: Text-Anchored Supervised Multimodal Contrastive Learning for fMRI-Based Brain Disorder Classification](https://mlsanity.com/paper/2610.05880/) (2026)

## Abstract (arXiv)

Understanding neurological disorder is a fundamental problem in neuroscience, which often requires the analysis of brain networks derived from functional magnetic resonance imaging (fMRI) data. Despite the prevalence of Graph Neural Networks (GNNs) and Graph Transformers in various domains, applying them to brain networks faces challenges. Specifically, the datasets are severely impacted by the noises caused by distribution shifts across sub-populations and the neglect of node identities, both obstruct the identification of disease-specific patterns. To tackle these challenges, we propose Contrasformer, a novel contrastive brain network Transformer. It generates a prior-knowledge-enhanced contrast graph to address the distribution shifts across sub-populations by a two-stream attention mechanism. A cross attention with identity embedding highlights the identity of nodes, and three auxiliary losses ensure group consistency. Evaluated on 4 functional brain network datasets over 4 different diseases, Contrasformer outperforms the state-of-the-art methods for brain networks by achieving up to 10.8% improvement in accuracy, which demonstrates its efficacy in neurological disorder identification. Case studies illustrate its interpretability, especially in the context of neuroscience. This paper provides a solution for analyzing brain networks, offering valuable insights into neurological disorders. Our code is available at urlhttps://github.com/AngusMonroe/Contrasformer.