# G-MIND: An End-to-End Multimodal Imaging-Genetics Framework for Biomarker Identification and Disease Classification

arXiv: https://arxiv.org/abs/2101.11656 · HTML: https://mlsanity.com/paper/2101.11656/

## Key facts

- Cited by 7 papers in the map (data through 14 Oct 2026).
- Builds on 6 of its 30 references that are in the map.
- First posted on arXiv 27 Jan 2021.

## Builds on (most cited first, 6 of 6)

- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [Dropout as a Bayesian Approximation: Representing Model Uncertainty in Deep Learning](https://mlsanity.com/paper/1506.02142/) (2015)
- [Delving Deep into Rectifiers: Surpassing Human-Level Performance on ImageNet Classification](https://mlsanity.com/paper/1502.01852/) (2015)
- [Categorical Reparameterization with Gumbel-Softmax](https://mlsanity.com/paper/1611.01144/) (2016)
- [Concrete Dropout](https://mlsanity.com/paper/1705.07832/) (2017)
- [Multimodal deep learning approach for joint EEG-EMG data compression and classification](https://mlsanity.com/paper/1703.08970/) (2017)

## Built on by (most cited first, 7 of 7)

- [Deep Multi-modal Fusion of Image and Non-image Data in Disease Diagnosis and Prognosis: A Review](https://mlsanity.com/paper/2203.15588/) (2022)
- [Review of multimodal machine learning approaches in healthcare](https://mlsanity.com/paper/2402.02460/) (2024)
- [Survival Prediction of Brain Cancer with Incomplete Radiology, Pathology, Genomics, and Demographic Data](https://mlsanity.com/paper/2203.04419/) (2022)
- [Multi-modality fusion using canonical correlation analysis methods: Application in breast cancer survival prediction from histology and genomics](https://mlsanity.com/paper/2111.13987/) (2021)
- [Learning Explainable Imaging-Genetics Associations Related to a Neurological Disorder](https://mlsanity.com/paper/2508.18303/) (2025)
- [AdaFuse: Adaptive Multimodal Fusion for Lung Cancer Risk Prediction via Reinforcement Learning](https://mlsanity.com/paper/2602.00347/) (2026)
- [CALM: Interpretable Cross-Modal Alignment for Biomarker Discovery from Unpaired Data](https://mlsanity.com/paper/2607.01656/) (2026)

## Abstract (arXiv)

We propose a novel deep neural network architecture to integrate imaging and genetics data, as guided by diagnosis, that provides interpretable biomarkers. Our model consists of an encoder, a decoder and a classifier. The encoder learns a non-linear subspace shared between the input data modalities. The classifier and the decoder act as regularizers to ensure that the low-dimensional encoding captures predictive differences between patients and controls. We use a learnable dropout layer to extract interpretable biomarkers from the data, and our unique training strategy can easily accommodate missing data modalities across subjects. We have evaluated our model on a population study of schizophrenia that includes two functional MRI (fMRI) paradigms and Single Nucleotide Polymorphism (SNP) data. Using 10-fold cross validation, we demonstrate that our model achieves better classification accuracy than baseline methods, and that this performance generalizes to a second dataset collected at a different site. In an exploratory analysis we further show that the biomarkers identified by our model are closely associated with the well-documented deficits in schizophrenia.