# Fusing Structural and Functional Connectivities using Disentangled VAE for Detecting MCI

arXiv: https://arxiv.org/abs/2306.09629 · HTML: https://mlsanity.com/paper/2306.09629/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 15 of its 48 references that are in the map.
- First posted on arXiv 16 Jun 2023.

## Builds on (most cited first, 15 of 15)

- [An Introduction to Variational Autoencoders](https://mlsanity.com/paper/1906.02691/) (2019)
- [Diffusion-Convolutional Neural Networks](https://mlsanity.com/paper/1511.02136/) (2015)
- [NiftyNet: a deep-learning platform for medical imaging](https://mlsanity.com/paper/1709.03485/) (2017)
- [Bidirectional Mapping Generative Adversarial Networks for Brain MR to PET Synthesis](https://mlsanity.com/paper/2008.03483/) (2020)
- [Fine Perceptive GANs for Brain MR Image Super-Resolution in Wavelet Domain](https://mlsanity.com/paper/2011.04145/) (2020)
- [Source-free unsupervised domain adaptation for cross-modality abdominal multi-organ segmentation](https://mlsanity.com/paper/2111.12221/) (2021)
- [Tensorizing GAN with High-Order Pooling for Alzheimer's Disease Assessment](https://mlsanity.com/paper/2008.00748/) (2020)
- [Multimodal Representations Learning and Adversarial Hypergraph Fusion for Early Alzheimer's Disease Prediction](https://mlsanity.com/paper/2107.09928/) (2021)
- [Multi-View Graph Convolutional Network and Its Applications on Neuroimage Analysis for Parkinson's Disease](https://mlsanity.com/paper/1805.08801/) (2018)
- [Unsupervised domain adaptation for cross-modality liver segmentation via joint adversarial learning and self-learning](https://mlsanity.com/paper/2109.05664/) (2021)
- [Morphological feature visualization of Alzheimer's disease via Multidirectional Perception GAN](https://mlsanity.com/paper/2111.12886/) (2021)
- [Brain Structure-Function Fusing Representation Learning using Adversarial Decomposed-VAE for Analyzing MCI](https://mlsanity.com/paper/2305.14404/) (2023)
- [A Prior Guided Adversarial Representation Learning and Hypergraph Perceptual Network for Predicting Abnormal Connections of Alzheimer's Disease](https://mlsanity.com/paper/2110.09302/) (2021)
- [Multiscale Autoencoder with Structural-Functional Attention Network for Alzheimer's Disease Prediction](https://mlsanity.com/paper/2208.04945/) (2022)
- [Brain Imaging-to-Graph Generation using Adversarial Hierarchical Diffusion Models for MCI Causality Analysis](https://mlsanity.com/paper/2305.10754/) (2023)

## Abstract (arXiv)

Brain network analysis is a useful approach to studying human brain disorders because it can distinguish patients from healthy people by detecting abnormal connections. Due to the complementary information from multiple modal neuroimages, multimodal fusion technology has a lot of potential for improving prediction performance. However, effective fusion of multimodal medical images to achieve complementarity is still a challenging problem. In this paper, a novel hierarchical structural-functional connectivity fusing (HSCF) model is proposed to construct brain structural-functional connectivity matrices and predict abnormal brain connections based on functional magnetic resonance imaging (fMRI) and diffusion tensor imaging (DTI). Specifically, the prior knowledge is incorporated into the separators for disentangling each modality of information by the graph convolutional networks (GCN). And a disentangled cosine distance loss is devised to ensure the disentanglement's effectiveness. Moreover, the hierarchical representation fusion module is designed to effectively maximize the combination of relevant and effective features between modalities, which makes the generated structural-functional connectivity more robust and discriminative in the cognitive disease analysis. Results from a wide range of tests performed on the public Alzheimer's Disease Neuroimaging Initiative (ADNI) database show that the proposed model performs better than competing approaches in terms of classification evaluation. In general, the proposed HSCF model is a promising model for generating brain structural-functional connectivities and identifying abnormal brain connections as cognitive disease progresses.