# Integrated Brain Connectivity Analysis with fMRI, DTI, and sMRI Powered by Interpretable Graph Neural Networks

arXiv: https://arxiv.org/abs/2408.14254 · HTML: https://mlsanity.com/paper/2408.14254/

## Key facts

- Cited by 12 papers in the map (data through 14 Oct 2026).
- Builds on 10 of its 81 references that are in the map.
- First posted on arXiv 26 Aug 2024.

## Builds on (most cited first, 10 of 10)

- [Graph Attention Networks](https://mlsanity.com/paper/1710.10903/) (2017)
- [How Powerful are Graph Neural Networks?](https://mlsanity.com/paper/1810.00826/) (2018)
- [Grad-CAM++: Improved Visual Explanations for Deep Convolutional Networks](https://mlsanity.com/paper/1710.11063/) (2017)
- [Attention Bottlenecks for Multimodal Fusion](https://mlsanity.com/paper/2107.00135/) (2021)
- [Ensemble manifold based regularized multi-modal graph convolutional network for cognitive ability prediction](https://mlsanity.com/paper/2101.08316/) (2021)
- [Interpretable multimodal fusion networks reveal mechanisms of brain cognition](https://mlsanity.com/paper/2006.09454/) (2020)
- [Latent Similarity Identifies Important Functional Connections for Phenotype Prediction](https://mlsanity.com/paper/2208.14484/) (2022)
- [Distance Correlation Based Brain Functional Connectivity Estimation and Non-Convex Multi-Task Learning for Developmental fMRI Studies](https://mlsanity.com/paper/2010.00116/) (2020)
- [An Interpretable Cross-Attentive Multi-modal MRI Fusion Framework for Schizophrenia Diagnosis](https://mlsanity.com/paper/2404.00144/) (2024)
- [Angle Basis: a Generative Model and Decomposition for Functional Connectivity](https://mlsanity.com/paper/2305.10541/) (2023)

## Built on by (most cited first, 12 of 12)

- [Multi-Atlas Brain Network Classification through Consistency Distillation and Complementary Information Fusion](https://mlsanity.com/paper/2410.08228/) (2024)
- [BrainMAP: Multimodal Graph Learning For Efficient Brain Disease Localization](https://mlsanity.com/paper/2506.11178/) (2025)
- [An Interpretable Representation Learning Approach for Diffusion Tensor Imaging](https://mlsanity.com/paper/2505.19110/) (2025)
- [Interpretable Neuropsychiatric Diagnosis via Concept-Guided Graph Neural Networks](https://mlsanity.com/paper/2510.03351/) (2025)
- [Beyond Pairwise Connections: Extracting High-Order Functional Brain Network Structures under Global Constraints](https://mlsanity.com/paper/2510.09175/) (2025)
- [Interpretable graph-based models on multimodal biomedical data integration: A technical review and benchmarking](https://mlsanity.com/paper/2505.01696/) (2025)
- [Predicting Cognition from fMRI:A Comparative Study of Graph, Transformer, and Kernel Models Across Task and Rest Conditions](https://mlsanity.com/paper/2507.21016/) (2025)
- [Uncovering Latent Communication Patterns in Brain Networks via Adaptive Flow Routing](https://mlsanity.com/paper/2602.00561/) (2026)
- [Supervised Deep Multimodal Matrix Factorization for Interpretable Brain Network Analysis](https://mlsanity.com/paper/2605.13312/) (2026)
- [GMN4AD: Graph Matching Network for Alzheimer's Disease Diagnosis with Test-Time Domain Adaptation using Multi-centered Structure Magnetic Resonance Imaging](https://mlsanity.com/paper/2606.13919/) (2026)
- [DynFS-MoE: Dynamic Functional-Structural Mixture-of-Experts for Post-Traumatic Epilepsy Diagnosis](https://mlsanity.com/paper/2606.16203/) (2026)
- [HyperAMS-Net: Adaptive Multi-Scale Spatial Hypergraph Network for Brain Disorder Classification](https://mlsanity.com/paper/2609.19755/) (2026)

## Abstract (arXiv)

Multimodal neuroimaging modeling has becomes a widely used approach but confronts considerable challenges due to heterogeneity, which encompasses variability in data types, scales, and formats across modalities. This variability necessitates the deployment of advanced computational methods to integrate and interpret these diverse datasets within a cohesive analytical framework. In our research, we amalgamate functional magnetic resonance imaging, diffusion tensor imaging, and structural MRI into a cohesive framework. This integration capitalizes on the unique strengths of each modality and their inherent interconnections, aiming for a comprehensive understanding of the brain's connectivity and anatomical characteristics. Utilizing the Glasser atlas for parcellation, we integrate imaging derived features from various modalities: functional connectivity from fMRI, structural connectivity from DTI, and anatomical features from sMRI within consistent regions. Our approach incorporates a masking strategy to differentially weight neural connections, thereby facilitating a holistic amalgamation of multimodal imaging data. This technique enhances interpretability at connectivity level, transcending traditional analyses centered on singular regional attributes. The model is applied to the Human Connectome Project's Development study to elucidate the associations between multimodal imaging and cognitive functions throughout youth. The analysis demonstrates improved predictive accuracy and uncovers crucial anatomical features and essential neural connections, deepening our understanding of brain structure and function.