# Fusing Modalities by Multiplexed Graph Neural Networks for Outcome Prediction in Tuberculosis

arXiv: https://arxiv.org/abs/2210.14377 · HTML: https://mlsanity.com/paper/2210.14377/

## Key facts

- Cited by 6 papers in the map (data through 14 Oct 2026).
- Builds on 9 of its 31 references that are in the map.
- First posted on arXiv 25 Oct 2022.

## Builds on (most cited first, 9 of 9)

- [Semi-Supervised Classification with Graph Convolutional Networks](https://mlsanity.com/paper/1609.02907/) (2016)
- [Densely Connected Convolutional Networks](https://mlsanity.com/paper/1608.06993/) (2016)
- [How Powerful are Graph Neural Networks?](https://mlsanity.com/paper/1810.00826/) (2018)
- [Modeling Relational Data with Graph Convolutional Networks](https://mlsanity.com/paper/1703.06103/) (2017)
- [Multimodal Machine Learning: A Survey and Taxonomy](https://mlsanity.com/paper/1705.09406/) (2017)
- [On Explainability of Graph Neural Networks via Subgraph Explorations](https://mlsanity.com/paper/2102.05152/) (2021)
- [Multilayer Networks](https://mlsanity.com/paper/1309.7233/) (2013)
- [Spatially Aware Multimodal Transformers for TextVQA](https://mlsanity.com/paper/2007.12146/) (2020)
- [Multimodal fusion of imaging and genomics for lung cancer recurrence prediction](https://mlsanity.com/paper/2002.01982/) (2020)

## Built on by (most cited first, 6 of 6)

- [Merging-Diverging Hybrid Transformer Networks for Survival Prediction in Head and Neck Cancer](https://mlsanity.com/paper/2307.03427/) (2023)
- [AdaMSS: Adaptive Multi-Modality Segmentation-to-Survival Learning for Survival Outcome Prediction from PET/CT Images](https://mlsanity.com/paper/2305.09946/) (2023)
- [MaxCorrMGNN: A Multi-Graph Neural Network Framework for Generalized Multimodal Fusion of Medical Data for Outcome Prediction](https://mlsanity.com/paper/2307.07093/) (2023)
- [Functional Graph Convolutional Networks: A unified multi-task and multi-modal learning framework to facilitate health and social-care insights](https://mlsanity.com/paper/2403.10158/) (2024)
- [Optimizing contrastive learning for cortical folding pattern detection](https://mlsanity.com/paper/2401.18035/) (2024)
- [Seeking Necessary and Sufficient Information from Multimodal Medical Data](https://mlsanity.com/paper/2603.00289/) (2026)

## Abstract (arXiv)

In a complex disease such as tuberculosis, the evidence for the disease and its evolution may be present in multiple modalities such as clinical, genomic, or imaging data. Effective patient-tailored outcome prediction and therapeutic guidance will require fusing evidence from these modalities. Such multimodal fusion is difficult since the evidence for the disease may not be uniform across all modalities, not all modality features may be relevant, or not all modalities may be present for all patients. All these nuances make simple methods of early, late, or intermediate fusion of features inadequate for outcome prediction. In this paper, we present a novel fusion framework using multiplexed graphs and derive a new graph neural network for learning from such graphs. Specifically, the framework allows modalities to be represented through their targeted encodings, and models their relationship explicitly via multiplexed graphs derived from salient features in a combined latent space. We present results that show that our proposed method outperforms state-of-the-art methods of fusing modalities for multi-outcome prediction on a large Tuberculosis (TB) dataset.