# U-GAT: Multimodal Graph Attention Network for COVID-19 Outcome Prediction

arXiv: https://arxiv.org/abs/2108.00860 · HTML: https://mlsanity.com/paper/2108.00860/

## Key facts

- Cited by 4 papers in the map (data through 14 Oct 2026).
- Builds on 27 of its 70 references that are in the map.
- First posted on arXiv 29 Jul 2021.

## Builds on (most cited first, 27 of 27)

- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [PyTorch: An Imperative Style, High-Performance Deep Learning Library](https://mlsanity.com/paper/1912.01703/) (2019)
- [U-Net: Convolutional Networks for Biomedical Image Segmentation](https://mlsanity.com/paper/1505.04597/) (2015)
- [Graph Attention Networks](https://mlsanity.com/paper/1710.10903/) (2017)
- [Inductive Representation Learning on Large Graphs](https://mlsanity.com/paper/1706.02216/) (2017)
- [XGBoost: A Scalable Tree Boosting System](https://mlsanity.com/paper/1603.02754/) (2016)
- [Fast Graph Representation Learning with PyTorch Geometric](https://mlsanity.com/paper/1903.02428/) (2019)
- [V-Net: Fully Convolutional Neural Networks for Volumetric Medical Image Segmentation](https://mlsanity.com/paper/1606.04797/) (2016)
- [Convolutional Neural Networks on Graphs with Fast Localized Spectral Filtering](https://mlsanity.com/paper/1606.09375/) (2016)
- [SqueezeNet: AlexNet-level accuracy with 50x fewer parameters and <0.5MB model size](https://mlsanity.com/paper/1602.07360/) (2016)
- [Disease Prediction using Graph Convolutional Networks: Application to Autism Spectrum Disorder and Alzheimer's Disease](https://mlsanity.com/paper/1806.01738/) (2018)
- [JCS: An Explainable COVID-19 Diagnosis System by Joint Classification and Segmentation](https://mlsanity.com/paper/2004.07054/) (2020)
- [Spectral Graph Convolutions for Population-based Disease Prediction](https://mlsanity.com/paper/1703.03020/) (2017)
- [Y-Net: Joint Segmentation and Classification for Diagnosis of Breast Biopsy Images](https://mlsanity.com/paper/1806.01313/) (2018)
- [Latent-Graph Learning for Disease Prediction](https://mlsanity.com/paper/2003.13620/) (2020)
- [Bootstrapping Graph Convolutional Neural Networks for Autism Spectrum Disorder Classification](https://mlsanity.com/paper/1704.07487/) (2017)
- [Graph Convolutional Networks for Coronary Artery Segmentation in Cardiac CT Angiography](https://mlsanity.com/paper/1908.05343/) (2019)
- [COVID_MTNet: COVID-19 Detection with Multi-Task Deep Learning Approaches](https://mlsanity.com/paper/2004.03747/) (2020)
- [A Novel Multi-task Deep Learning Model for Skin Lesion Segmentation and Classification](https://mlsanity.com/paper/1703.01025/) (2017)
- [Synergistic Learning of Lung Lobe Segmentation and Hierarchical Multi-Instance Classification for Automated Severity Assessment of COVID-19 in CT Images](https://mlsanity.com/paper/2005.03832/) (2020)
- [CT-based COVID-19 Triage: Deep Multitask Learning Improves Joint Identification and Severity Quantification](https://mlsanity.com/paper/2006.01441/) (2020)
- [Multitask Classification and Segmentation for Cancer Diagnosis in Mammography](https://mlsanity.com/paper/1909.05397/) (2019)
- [Zoom in to where it matters: a hierarchical graph based model for mammogram analysis](https://mlsanity.com/paper/1912.07517/) (2019)
- [Hypergraph Learning for Identification of COVID-19 with CT Imaging](https://mlsanity.com/paper/2005.04043/) (2020)
- [Graph-based Pyramid Global Context Reasoning with a Saliency-aware Projection for COVID-19 Lung Infections Segmentation](https://mlsanity.com/paper/2103.04235/) (2021)
- [Adaptive Image-Feature Learning for Disease Classification Using Inductive Graph Networks](https://mlsanity.com/paper/1905.03036/) (2019)
- [Integrative Analysis for COVID-19 Patient Outcome Prediction](https://mlsanity.com/paper/2007.10416/) (2020)

## Built on by (most cited first, 4 of 4)

- [Dynamic Multimodal Information Bottleneck for Multimodality Classification](https://mlsanity.com/paper/2311.01066/) (2023)
- [Hospital transfer risk prediction for COVID-19 patients from a medicalized hotel based on Diffusion GraphSAGE](https://mlsanity.com/paper/2301.01596/) (2022)
- [Connecting the Dots: Graph Neural Network Powered Ensemble and Classification of Medical Images](https://mlsanity.com/paper/2311.07321/) (2023)
- [Interpretable graph-based models on multimodal biomedical data integration: A technical review and benchmarking](https://mlsanity.com/paper/2505.01696/) (2025)

## Abstract (arXiv)

During the first wave of COVID-19, hospitals were overwhelmed with the high number of admitted patients. An accurate prediction of the most likely individual disease progression can improve the planning of limited resources and finding the optimal treatment for patients. However, when dealing with a newly emerging disease such as COVID-19, the impact of patient- and disease-specific factors (e.g. body weight or known co-morbidities) on the immediate course of disease is by and large unknown. In the case of COVID-19, the need for intensive care unit (ICU) admission of pneumonia patients is often determined only by acute indicators such as vital signs (e.g. breathing rate, blood oxygen levels), whereas statistical analysis and decision support systems that integrate all of the available data could enable an earlier prognosis. To this end, we propose a holistic graph-based approach combining both imaging and non-imaging information. Specifically, we introduce a multimodal similarity metric to build a population graph for clustering patients and an image-based end-to-end Graph Attention Network to process this graph and predict the COVID-19 patient outcomes: admission to ICU, need for ventilation and mortality. Additionally, the network segments chest CT images as an auxiliary task and extracts image features and radiomics for feature fusion with the available metadata. Results on a dataset collected in Klinikum rechts der Isar in Munich, Germany show that our approach outperforms single modality and non-graph baselines. Moreover, our clustering and graph attention allow for increased understanding of the patient relationships within the population graph and provide insight into the network's decision-making process.