# Fusing multimodal neuroimaging data with a variational autoencoder

arXiv: https://arxiv.org/abs/2105.01128 · HTML: https://mlsanity.com/paper/2105.01128/

## Key facts

- Cited by 2 papers in the map (data through 14 Oct 2026).
- Builds on 7 of its 22 references that are in the map.
- First posted on arXiv 3 May 2021.

## Builds on (most cited first, 7 of 7)

- [PyTorch: An Imperative Style, High-Performance Deep Learning Library](https://mlsanity.com/paper/1912.01703/) (2019)
- [Auto-Encoding Variational Bayes](https://mlsanity.com/paper/1312.6114/) (2013)
- [Variational Mixture-of-Experts Autoencoders for Multi-Modal Deep Generative Models](https://mlsanity.com/paper/1911.03393/) (2019)
- [TorchIO: A Python library for efficient loading, preprocessing, augmentation and patch-based sampling of medical images in deep learning](https://mlsanity.com/paper/2003.04696/) (2020)
- [Self-Supervised Multimodal Domino: in Search of Biomarkers for Alzheimer's Disease](https://mlsanity.com/paper/2012.13623/) (2020)
- [Tasting the cake: evaluating self-supervised generalization on out-of-distribution multimodal MRI data](https://mlsanity.com/paper/2103.15914/) (2021)
- [Multidataset Independent Subspace Analysis with Application to Multimodal Fusion](https://mlsanity.com/paper/1911.04048/) (2019)

## Built on by (most cited first, 2 of 2)

- [Modern Views of Machine Learning for Precision Psychiatry](https://mlsanity.com/paper/2204.01607/) (2022)
- [Generalized Product-of-Experts for Learning Multimodal Representations in Noisy Environments](https://mlsanity.com/paper/2211.03587/) (2022)

## Abstract (arXiv)

Neuroimaging studies often involve the collection of multiple data modalities. These modalities contain both shared and mutually exclusive information about the brain. This work aims at finding a scalable and interpretable method to fuse the information of multiple neuroimaging modalities using a variational autoencoder (VAE). To provide an initial assessment, this work evaluates the representations that are learned using a schizophrenia classification task. A support vector machine trained on the representations achieves an area under the curve for the classifier's receiver operating characteristic (ROC-AUC) of 0.8610.