# Heterogeneous Graph Learning for Multi-modal Medical Data Analysis

arXiv: https://arxiv.org/abs/2211.15158 · HTML: https://mlsanity.com/paper/2211.15158/

## Key facts

- Cited by 16 papers in the map (data through 14 Oct 2026).
- Builds on 27 of its 65 references that are in the map.
- First posted on arXiv 28 Nov 2022.

## Builds on (most cited first, 27 of 27)

- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [A Simple Framework for Contrastive Learning of Visual Representations](https://mlsanity.com/paper/2002.05709/) (2020)
- [Semi-Supervised Classification with Graph Convolutional Networks](https://mlsanity.com/paper/1609.02907/) (2016)
- [Graph Attention Networks](https://mlsanity.com/paper/1710.10903/) (2017)
- [Momentum Contrast for Unsupervised Visual Representation Learning](https://mlsanity.com/paper/1911.05722/) (2019)
- [Inductive Representation Learning on Large Graphs](https://mlsanity.com/paper/1706.02216/) (2017)
- [Modeling Relational Data with Graph Convolutional Networks](https://mlsanity.com/paper/1703.06103/) (2017)
- [Unsupervised Representation Learning by Predicting Image Rotations](https://mlsanity.com/paper/1803.07728/) (2018)
- [Unsupervised Learning of Visual Representations by Solving Jigsaw Puzzles](https://mlsanity.com/paper/1603.09246/) (2016)
- [Unsupervised Visual Representation Learning by Context Prediction](https://mlsanity.com/paper/1505.05192/) (2015)
- [Heterogeneous Graph Attention Network](https://mlsanity.com/paper/1903.07293/) (2019)
- [Deep Graph Contrastive Representation Learning](https://mlsanity.com/paper/2006.04131/) (2020)
- [Big Self-Supervised Models Advance Medical Image Classification](https://mlsanity.com/paper/2101.05224/) (2021)
- [3D Self-Supervised Methods for Medical Imaging](https://mlsanity.com/paper/2006.03829/) (2020)
- [A Survey of Heterogeneous Information Network Analysis](https://mlsanity.com/paper/1511.04854/) (2015)
- [Unsupervised Attributed Multiplex Network Embedding](https://mlsanity.com/paper/1911.06750/) (2019)
- [Augmentation-Free Self-Supervised Learning on Graphs](https://mlsanity.com/paper/2112.02472/) (2021)
- [Representation Learning for Attributed Multiplex Heterogeneous Network](https://mlsanity.com/paper/1905.01669/) (2019)
- [HDMI: High-order Deep Multiplex Infomax](https://mlsanity.com/paper/2102.07810/) (2021)
- [3D Deep Learning on Medical Images: A Review](https://mlsanity.com/paper/2004.00218/) (2020)
- [Spectral Graph Convolutions for Population-based Disease Prediction](https://mlsanity.com/paper/1703.03020/) (2017)
- [InceptionGCN: Receptive Field Aware Graph Convolutional Network for Disease Prediction](https://mlsanity.com/paper/1903.04233/) (2019)
- [Deep Multi-modal Fusion of Image and Non-image Data in Disease Diagnosis and Prognosis: A Review](https://mlsanity.com/paper/2203.15588/) (2022)
- [An Attention-based Collaboration Framework for Multi-View Network Representation Learning](https://mlsanity.com/paper/1709.06636/) (2017)
- [Relational Self-Supervised Learning on Graphs](https://mlsanity.com/paper/2208.10493/) (2022)
- [GraFN: Semi-Supervised Node Classification on Graph with Few Labels via Non-Parametric Distribution Assignment](https://mlsanity.com/paper/2204.01303/) (2022)
- [Context Matters: Graph-based Self-supervised Representation Learning for Medical Images](https://mlsanity.com/paper/2012.06457/) (2020)

## Built on by (most cited first, 16 of 16)

- [Learning on Multimodal Graphs: A Survey](https://mlsanity.com/paper/2402.05322/) (2024)
- [Multimodal Data Integration for Precision Oncology: Challenges and Future Directions](https://mlsanity.com/paper/2406.19611/) (2024)
- [Task-Equivariant Graph Few-shot Learning](https://mlsanity.com/paper/2305.18758/) (2023)
- [AliFuse: Aligning and Fusing Multi-modal Medical Data for Computer-Aided Diagnosis](https://mlsanity.com/paper/2401.01074/) (2024)
- [GTP-4o: Modality-prompted Heterogeneous Graph Learning for Omni-modal Biomedical Representation](https://mlsanity.com/paper/2407.05540/) (2024)
- [HoloDx: Knowledge- and Data-Driven Multimodal Diagnosis of Alzheimer's Disease](https://mlsanity.com/paper/2504.19075/) (2025)
- [FOAA: Flattened Outer Arithmetic Attention For Multimodal Tumor Classification](https://mlsanity.com/paper/2403.06339/) (2024)
- [Compact & Capable: Harnessing Graph Neural Networks and Edge Convolution for Medical Image Classification](https://mlsanity.com/paper/2307.12790/) (2023)
- [Uncertainty-Aware Hardware Trojan Detection Using Multimodal Deep Learning](https://mlsanity.com/paper/2401.09479/) (2024)
- [Representation Learning with Mutual Influence of Modalities for Node Classification in Multi-Modal Heterogeneous Networks](https://mlsanity.com/paper/2505.07895/) (2025)
- [LeMoF: Level-guided Multimodal Fusion for Heterogeneous Clinical Data](https://mlsanity.com/paper/2601.10092/) (2026)
- [Modality as Heterogeneity: Node Splitting and Graph Rewiring for Multimodal Graph Learning](https://mlsanity.com/paper/2602.00067/) (2026)
- [GAMMA-PD: Graph-based Analysis of Multi-Modal Motor Impairment Assessments in Parkinson's Disease](https://mlsanity.com/paper/2410.00944/) (2024)
- [Multimodal Sheaf-based Network for Glioblastoma Molecular Subtype Prediction](https://mlsanity.com/paper/2508.09717/) (2025)
- [Bridging Graph and State-Space Modeling for Intensive Care Unit Length of Stay Prediction](https://mlsanity.com/paper/2508.17554/) (2025)
- [MRI-based Deep Radiomic Phenotyping of Neuromuscular Disorders: A Topology-driven Characterization](https://mlsanity.com/paper/2608.24415/) (2026)

## Abstract (arXiv)

Routine clinical visits of a patient produce not only image data, but also non-image data containing clinical information regarding the patient, i.e., medical data is multi-modal in nature. Such heterogeneous modalities offer different and complementary perspectives on the same patient, resulting in more accurate clinical decisions when they are properly combined. However, despite its significance, how to effectively fuse the multi-modal medical data into a unified framework has received relatively little attention. In this paper, we propose an effective graph-based framework called HetMed (Heterogeneous Graph Learning for Multi-modal Medical Data Analysis) for fusing the multi-modal medical data. Specifically, we construct a multiplex network that incorporates multiple types of non-image features of patients to capture the complex relationship between patients in a systematic way, which leads to more accurate clinical decisions. Extensive experiments on various real-world datasets demonstrate the superiority and practicality of HetMed. The source code for HetMed is available at https://github.com/Sein-Kim/Multimodal-Medical.