# Ensemble manifold based regularized multi-modal graph convolutional network for cognitive ability prediction

arXiv: https://arxiv.org/abs/2101.08316 · HTML: https://mlsanity.com/paper/2101.08316/

## Key facts

- Cited by 9 papers in the map (data through 14 Oct 2026).
- Builds on 8 of its 50 references that are in the map.
- First posted on arXiv 20 Jan 2021.

## Builds on (most cited first, 8 of 8)

- [Semi-Supervised Classification with Graph Convolutional Networks](https://mlsanity.com/paper/1609.02907/) (2016)
- [Grad-CAM: Visual Explanations from Deep Networks via Gradient-based Localization](https://mlsanity.com/paper/1610.02391/) (2016)
- [Spectral Networks and Locally Connected Networks on Graphs](https://mlsanity.com/paper/1312.6203/) (2013)
- [Grad-CAM++: Improved Visual Explanations for Deep Convolutional Networks](https://mlsanity.com/paper/1710.11063/) (2017)
- [Disease Prediction using Graph Convolutional Networks: Application to Autism Spectrum Disorder and Alzheimer's Disease](https://mlsanity.com/paper/1806.01738/) (2018)
- [Graph Neural Network for Interpreting Task-fMRI Biomarkers](https://mlsanity.com/paper/1907.01661/) (2019)
- [Multi-View Graph Convolutional Network and Its Applications on Neuroimage Analysis for Parkinson's Disease](https://mlsanity.com/paper/1805.08801/) (2018)
- [Diabetic Retinopathy Detection via Deep Convolutional Networks for Discriminative Localization and Visual Explanation](https://mlsanity.com/paper/1703.10757/) (2017)

## Built on by (most cited first, 9 of 9)

- [Integrated Brain Connectivity Analysis with fMRI, DTI, and sMRI Powered by Interpretable Graph Neural Networks](https://mlsanity.com/paper/2408.14254/) (2024)
- [A Demographic-Conditioned Variational Autoencoder for fMRI Distribution Sampling and Removal of Confounds](https://mlsanity.com/paper/2405.07977/) (2024)
- [Applications of interpretable deep learning in neuroimaging: a comprehensive review](https://mlsanity.com/paper/2406.17792/) (2024)
- [Multimodal Graph Representation Learning for Robust Surgical Workflow Recognition with Adversarial Feature Disentanglement](https://mlsanity.com/paper/2505.01766/) (2025)
- [A Learnable Counter-condition Analysis Framework for Functional Connectivity-based Neurological Disorder Diagnosis](https://mlsanity.com/paper/2310.03964/) (2023)
- [ImageNomer: description of a functional connectivity and omics analysis tool and case study identifying a race confound](https://mlsanity.com/paper/2302.00767/) (2023)
- [Exploring General Intelligence via Gated Graph Transformer in Functional Connectivity Studies](https://mlsanity.com/paper/2401.10348/) (2024)
- [Interpretable graph-based models on multimodal biomedical data integration: A technical review and benchmarking](https://mlsanity.com/paper/2505.01696/) (2025)
- [MethConvTransformer: A Deep Learning Framework for Cross-Tissue Alzheimer's Disease Detection](https://mlsanity.com/paper/2601.00143/) (2026)

## Abstract (arXiv)

Objective: Multi-modal functional magnetic resonance imaging (fMRI) can be used to make predictions about individual behavioral and cognitive traits based on brain connectivity networks. Methods: To take advantage of complementary information from multi-modal fMRI, we propose an interpretable multi-modal graph convolutional network (MGCN) model, incorporating the fMRI time series and the functional connectivity (FC) between each pair of brain regions. Specifically, our model learns a graph embedding from individual brain networks derived from multi-modal data. A manifold-based regularization term is then enforced to consider the relationships of subjects both within and between modalities. Furthermore, we propose the gradient-weighted regression activation mapping (Grad-RAM) and the edge mask learning to interpret the model, which is used to identify significant cognition-related biomarkers. Results: We validate our MGCN model on the Philadelphia Neurodevelopmental Cohort to predict individual wide range achievement test (WRAT) score. Our model obtains superior predictive performance over GCN with a single modality and other competing approaches. The identified biomarkers are cross-validated from different approaches. Conclusion and Significance: This paper develops a new interpretable graph deep learning framework for cognitive ability prediction, with the potential to overcome the limitations of several current data-fusion models. The results demonstrate the power of MGCN in analyzing multi-modal fMRI and discovering significant biomarkers for human brain studies.