# MaxCorrMGNN: A Multi-Graph Neural Network Framework for Generalized Multimodal Fusion of Medical Data for Outcome Prediction

arXiv: https://arxiv.org/abs/2307.07093 · HTML: https://mlsanity.com/paper/2307.07093/

## Key facts

- Cited by 2 papers in the map (data through 14 Oct 2026).
- Builds on 14 of its 27 references that are in the map.
- First posted on arXiv 13 Jul 2023.

## Builds on (most cited first, 14 of 14)

- [Semi-Supervised Classification with Graph Convolutional Networks](https://mlsanity.com/paper/1609.02907/) (2016)
- [Graph Attention Networks](https://mlsanity.com/paper/1710.10903/) (2017)
- [Densely Connected Convolutional Networks](https://mlsanity.com/paper/1608.06993/) (2016)
- [How Powerful are Graph Neural Networks?](https://mlsanity.com/paper/1810.00826/) (2018)
- [Multimodal Machine Learning: A Survey and Taxonomy](https://mlsanity.com/paper/1705.09406/) (2017)
- [Latent-Graph Learning for Disease Prediction](https://mlsanity.com/paper/2003.13620/) (2020)
- [Multi-modal Graph Learning for Disease Prediction](https://mlsanity.com/paper/2203.05880/) (2022)
- [An Efficient Approach to Informative Feature Extraction from Multimodal Data](https://mlsanity.com/paper/1811.08979/) (2018)
- [Multimodal fusion of imaging and genomics for lung cancer recurrence prediction](https://mlsanity.com/paper/2002.01982/) (2020)
- [Fusing Modalities by Multiplexed Graph Neural Networks for Outcome Prediction in Tuberculosis](https://mlsanity.com/paper/2210.14377/) (2022)
- [A Matrix Autoencoder Framework to Align the Functional and Structural Connectivity Manifolds as Guided by Behavioral Phenotypes](https://mlsanity.com/paper/2105.14409/) (2021)
- [Deep sr-DDL: Deep Structurally Regularized Dynamic Dictionary Learning to Integrate Multimodal and Dynamic Functional Connectomics data for Multidimensional Clinical Characterizations](https://mlsanity.com/paper/2008.12410/) (2020)
- [Multi-modality fusion using canonical correlation analysis methods: Application in breast cancer survival prediction from histology and genomics](https://mlsanity.com/paper/2111.13987/) (2021)
- [A Deep-Generative Hybrid Model to Integrate Multimodal and Dynamic Connectivity for Predicting Spectrum-Level Deficits in Autism](https://mlsanity.com/paper/2007.01931/) (2020)

## Built on by (most cited first, 2 of 2)

- [Navigating the landscape of multimodal AI in medicine: a scoping review on technical challenges and clinical applications](https://mlsanity.com/paper/2411.03782/) (2024)
- [Heterogeneous Graph Alignment for Joint Reasoning and Interpretability](https://mlsanity.com/paper/2601.22593/) (2026)

## Abstract (arXiv)

With the emergence of multimodal electronic health records, the evidence for an outcome may be captured across multiple modalities ranging from clinical to imaging and genomic data. Predicting outcomes effectively requires fusion frameworks capable of modeling fine-grained and multi-faceted complex interactions between modality features within and across patients. We develop an innovative fusion approach called MaxCorr MGNN that models non-linear modality correlations within and across patients through Hirschfeld-Gebelein-Renyi maximal correlation (MaxCorr) embeddings, resulting in a multi-layered graph that preserves the identities of the modalities and patients. We then design, for the first time, a generalized multi-layered graph neural network (MGNN) for task-informed reasoning in multi-layered graphs, that learns the parameters defining patient-modality graph connectivity and message passing in an end-to-end fashion. We evaluate our model an outcome prediction task on a Tuberculosis (TB) dataset consistently outperforming several state-of-the-art neural, graph-based and traditional fusion techniques.