# Deep Representation Learning For Multimodal Brain Networks

arXiv: https://arxiv.org/abs/2007.09777 · HTML: https://mlsanity.com/paper/2007.09777/

## Key facts

- Cited by 12 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 19 Jul 2020.

## Built on by (most cited first, 12 of 12)

- [Graph Neural Networks in Network Neuroscience](https://mlsanity.com/paper/2106.03535/) (2021)
- [Graph-Based Deep Learning for Medical Diagnosis and Analysis: Past, Present and Future](https://mlsanity.com/paper/2105.13137/) (2021)
- [A Survey on Graph Neural Networks and Graph Transformers in Computer Vision: A Task-Oriented Perspective](https://mlsanity.com/paper/2209.13232/) (2022)
- [Non-isomorphic Inter-modality Graph Alignment and Synthesis for Holistic Brain Mapping](https://mlsanity.com/paper/2107.06281/) (2021)
- [Contrastive Brain Network Learning via Hierarchical Signed Graph Pooling Model](https://mlsanity.com/paper/2207.07650/) (2022)
- [Graph Convolutional Networks for Multi-modality Medical Imaging: Methods, Architectures, and Clinical Applications](https://mlsanity.com/paper/2202.08916/) (2022)
- [A Matrix Autoencoder Framework to Align the Functional and Structural Connectivity Manifolds as Guided by Behavioral Phenotypes](https://mlsanity.com/paper/2105.14409/) (2021)
- [Heterogeneous Graph-Based Multimodal Brain Network Learning](https://mlsanity.com/paper/2110.08465/) (2021)
- [Modern Views of Machine Learning for Precision Psychiatry](https://mlsanity.com/paper/2204.01607/) (2022)
- [Graph-in-Graph (GiG): Learning interpretable latent graphs in non-Euclidean domain for biological and healthcare applications](https://mlsanity.com/paper/2204.00323/) (2022)
- [Generative AI Enables Structural Brain Network Construction from fMRI via Symmetric Diffusion Learning](https://mlsanity.com/paper/2309.16205/) (2023)
- [Functional2Structural: Cross-Modality Brain Networks Representation Learning](https://mlsanity.com/paper/2205.07854/) (2022)

## Abstract (arXiv)

Applying network science approaches to investigate the functions and anatomy of the human brain is prevalent in modern medical imaging analysis. Due to the complex network topology, for an individual brain, mining a discriminative network representation from the multimodal brain networks is non-trivial. The recent success of deep learning techniques on graph-structured data suggests a new way to model the non-linear cross-modality relationship. However, current deep brain network methods either ignore the intrinsic graph topology or require a network basis shared within a group. To address these challenges, we propose a novel end-to-end deep graph representation learning (Deep Multimodal Brain Networks - DMBN) to fuse multimodal brain networks. Specifically, we decipher the cross-modality relationship through a graph encoding and decoding process. The higher-order network mappings from brain structural networks to functional networks are learned in the node domain. The learned network representation is a set of node features that are informative to induce brain saliency maps in a supervised manner. We test our framework in both synthetic and real image data. The experimental results show the superiority of the proposed method over some other state-of-the-art deep brain network models.