# Learning on Multimodal Graphs: A Survey

arXiv: https://arxiv.org/abs/2402.05322 · HTML: https://mlsanity.com/paper/2402.05322/

## Key facts

- Cited by 22 papers in the map (data through 14 Oct 2026).
- Builds on 29 of its 62 references that are in the map.
- First posted on arXiv 7 Feb 2024.

## Builds on (most cited first, 29 of 29)

- [Graph Attention Networks](https://mlsanity.com/paper/1710.10903/) (2017)
- [Modeling Relational Data with Graph Convolutional Networks](https://mlsanity.com/paper/1703.06103/) (2017)
- [MME: A Comprehensive Evaluation Benchmark for Multimodal Large Language Models](https://mlsanity.com/paper/2306.13394/) (2023)
- [Simplifying Graph Convolutional Networks](https://mlsanity.com/paper/1902.07153/) (2019)
- [Graph Contrastive Learning with Augmentations](https://mlsanity.com/paper/2010.13902/) (2020)
- [Heterogeneous Graph Attention Network](https://mlsanity.com/paper/1903.07293/) (2019)
- [KGAT: Knowledge Graph Attention Network for Recommendation](https://mlsanity.com/paper/1905.07854/) (2019)
- [A Survey on Multimodal Large Language Models for Autonomous Driving](https://mlsanity.com/paper/2311.12320/) (2023)
- [Retrieval-Augmented Multimodal Language Modeling](https://mlsanity.com/paper/2211.12561/) (2022)
- [Learning Multimodal Graph-to-Graph Translation for Molecular Optimization](https://mlsanity.com/paper/1812.01070/) (2018)
- [GCNet: Graph Completion Network for Incomplete Multimodal Learning in Conversation](https://mlsanity.com/paper/2203.02177/) (2022)
- [Provable Dynamic Fusion for Low-Quality Multimodal Data](https://mlsanity.com/paper/2306.02050/) (2023)
- [BrainGB: A Benchmark for Brain Network Analysis with Graph Neural Networks](https://mlsanity.com/paper/2204.07054/) (2022)
- [Simple and Efficient Heterogeneous Graph Neural Network](https://mlsanity.com/paper/2207.02547/) (2022)
- [Image-embodied Knowledge Representation Learning](https://mlsanity.com/paper/1609.07028/) (2016)
- [Multimodal learning with graphs](https://mlsanity.com/paper/2209.03299/) (2022)
- [Differentiable Scaffolding Tree for Molecular Optimization](https://mlsanity.com/paper/2109.10469/) (2021)
- [Hybrid Transformer with Multi-level Fusion for Multimodal Knowledge Graph Completion](https://mlsanity.com/paper/2205.02357/) (2022)
- [Multimodal Analogical Reasoning over Knowledge Graphs](https://mlsanity.com/paper/2210.00312/) (2022)
- [FormNetV2: Multimodal Graph Contrastive Learning for Form Document Information Extraction](https://mlsanity.com/paper/2305.02549/) (2023)
- [Graph Neural Networks for Multimodal Single-Cell Data Integration](https://mlsanity.com/paper/2203.01884/) (2022)
- [Tagging before Alignment: Integrating Multi-Modal Tags for Video-Text Retrieval](https://mlsanity.com/paper/2301.12644/) (2023)
- [Heterogeneous Graph Learning for Multi-modal Medical Data Analysis](https://mlsanity.com/paper/2211.15158/) (2022)
- [Multimodal Learning on Graphs for Disease Relation Extraction](https://mlsanity.com/paper/2203.08893/) (2022)
- [MoSE: Modality Split and Ensemble for Multimodal Knowledge Graph Completion](https://mlsanity.com/paper/2210.08821/) (2022)
- [Transferring Pre-trained Multimodal Representations with Cross-modal Similarity Matching](https://mlsanity.com/paper/2301.02903/) (2023)
- [Joyful: Joint Modality Fusion and Graph Contrastive Learning for Multimodal Emotion Recognition](https://mlsanity.com/paper/2311.11009/) (2023)
- [Recipe2Vec: Multi-modal Recipe Representation Learning with Graph Neural Networks](https://mlsanity.com/paper/2205.12396/) (2022)
- [Multimodal Graph Transformer for Multimodal Question Answering](https://mlsanity.com/paper/2305.00581/) (2023)

## Built on by (most cited first, 22 of 22)

- [Graphs Meet AI Agents: Taxonomy, Progress, and Future Opportunities](https://mlsanity.com/paper/2506.18019/) (2025)
- [Cross-Contrastive Clustering for Multimodal Attributed Graphs with Dual Graph Filtering](https://mlsanity.com/paper/2511.20030/) (2025)
- [OpenMAG: A Comprehensive Benchmark for Multimodal-Attributed Graph](https://mlsanity.com/paper/2602.05576/) (2026)
- [MLaGA: Multimodal Large Language and Graph Assistant](https://mlsanity.com/paper/2506.02568/) (2025)
- [Multi-modal Generative AI: Multi-modal LLMs, Diffusions, and the Unification](https://mlsanity.com/paper/2409.14993/) (2024)
- [Toward Effective Multimodal Graph Foundation Model: A Divide-and-Conquer Based Approach](https://mlsanity.com/paper/2602.04116/) (2026)
- [Expressive and Scalable Quantum Fusion for Multimodal Learning](https://mlsanity.com/paper/2510.06938/) (2025)
- [Transformer-Based Multimodal Knowledge Graph Completion with Link-Aware Contexts](https://mlsanity.com/paper/2501.15688/) (2025)
- [Grounding the Ungrounded: A Spectral-Graph Framework for Quantifying Hallucinations in Multimodal LLMs](https://mlsanity.com/paper/2508.19366/) (2025)
- [Graph Integrated Multimodal Concept Bottleneck Model](https://mlsanity.com/paper/2510.00701/) (2025)
- [Towards Robust Federated Multimodal Graph Learning under Modality Heterogeneity](https://mlsanity.com/paper/2605.12584/) (2026)
- [Context-aware Modality-Topology Co-Alignment for Multimodal Attributed Graphs](https://mlsanity.com/paper/2606.14172/) (2026)
- [Towards Multimodal Graph Large Language Model](https://mlsanity.com/paper/2506.09738/) (2025)
- [Multimodal Sheaf-based Network for Glioblastoma Molecular Subtype Prediction](https://mlsanity.com/paper/2508.09717/) (2025)
- [Structure Matters: Brain Graph Augmentation via Learnable Edge Masking for Data-efficient Psychiatric Diagnosis](https://mlsanity.com/paper/2509.09744/) (2025)
- [Data-Efficient Psychiatric Disorder Detection via Self-supervised Learning on Frequency-enhanced Brain Networks](https://mlsanity.com/paper/2509.10524/) (2025)
- [SLIP: Structural-aware Language-Image Pretraining for Vision-Language Alignment](https://mlsanity.com/paper/2511.03019/) (2025)
- [Stable Multimodal Graph Unlearning via Feature-Dimension Aware Quantile Selection](https://mlsanity.com/paper/2605.03303/) (2026)
- [Beyond the Aggregation Dilemma: Prior-Retaining Decoupled Learning for Multimodal Graphs](https://mlsanity.com/paper/2605.24684/) (2026)
- [TAHB: A Comprehensive Benchmark for Text-Attributed Hypergraph Learning](https://mlsanity.com/paper/2608.15055/) (2026)
- [ICE: Task-Aligned Clifford Latent Fields for Multimodal Graph Foundation Models](https://mlsanity.com/paper/2609.29398/) (2026)
- [GraphSelect for Budgeted Representation Selection in Multimodal Graph Inference](https://mlsanity.com/paper/2609.33561/) (2026)

## Abstract (arXiv)

Multimodal data pervades various domains, including healthcare, social media, and transportation, where multimodal graphs play a pivotal role. Machine learning on multimodal graphs, referred to as multimodal graph learning (MGL), is essential for successful artificial intelligence (AI) applications. The burgeoning research in this field encompasses diverse graph data types and modalities, learning techniques, and application scenarios. This survey paper conducts a comparative analysis of existing works in multimodal graph learning, elucidating how multimodal learning is achieved across different graph types and exploring the characteristics of prevalent learning techniques. Additionally, we delineate significant applications of multimodal graph learning and offer insights into future directions in this domain. Consequently, this paper serves as a foundational resource for researchers seeking to comprehend existing MGL techniques and their applicability across diverse scenarios.