# Mosaic of Modalities: A Comprehensive Benchmark for Multimodal Graph Learning

arXiv: https://arxiv.org/abs/2406.16321 · HTML: https://mlsanity.com/paper/2406.16321/

## Key facts

- Cited by 29 papers in the map (data through 14 Oct 2026).
- Builds on 34 of its 53 references that are in the map.
- First posted on arXiv 24 Jun 2024.

## Builds on (most cited first, 30 of 34)

- [Learning Transferable Visual Models From Natural Language Supervision](https://mlsanity.com/paper/2103.00020/) (2021)
- [An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale](https://mlsanity.com/paper/2010.11929/) (2020)
- [Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformer](https://mlsanity.com/paper/1910.10683/) (2019)
- [Semi-Supervised Classification with Graph Convolutional Networks](https://mlsanity.com/paper/1609.02907/) (2016)
- [DINOv2: Learning Robust Visual Features without Supervision](https://mlsanity.com/paper/2304.07193/) (2023)
- [Inductive Representation Learning on Large Graphs](https://mlsanity.com/paper/1706.02216/) (2017)
- [Optuna: A Next-generation Hyperparameter Optimization Framework](https://mlsanity.com/paper/1907.10902/) (2019)
- [Open Graph Benchmark: Datasets for Machine Learning on Graphs](https://mlsanity.com/paper/2005.00687/) (2020)
- [ImageBind: One Embedding Space To Bind Them All](https://mlsanity.com/paper/2305.05665/) (2023)
- [Large Language Models](https://mlsanity.com/paper/2307.05782/) (2023)
- [Eyes Wide Shut? Exploring the Visual Shortcomings of Multimodal LLMs](https://mlsanity.com/paper/2401.06209/) (2024)
- [TUDataset: A collection of benchmark datasets for learning with graphs](https://mlsanity.com/paper/2007.08663/) (2020)
- [FLAVA: A Foundational Language And Vision Alignment Model](https://mlsanity.com/paper/2112.04482/) (2021)
- [Multimodal Learning with Transformers: A Survey](https://mlsanity.com/paper/2206.06488/) (2022)
- [Long Range Graph Benchmark](https://mlsanity.com/paper/2206.08164/) (2022)
- [Bridging Language and Items for Retrieval and Recommendation: Benchmarking LLMs as Semantic Encoders](https://mlsanity.com/paper/2403.03952/) (2024)
- [Large Language Models on Graphs: A Comprehensive Survey](https://mlsanity.com/paper/2312.02783/) (2023)
- [GOOD: A Graph Out-of-Distribution Benchmark](https://mlsanity.com/paper/2206.08452/) (2022)
- [MMKG: Multi-Modal Knowledge Graphs](https://mlsanity.com/paper/1903.05485/) (2019)
- [CoDEx: A Comprehensive Knowledge Graph Completion Benchmark](https://mlsanity.com/paper/2009.07810/) (2020)
- [Graph Neural Networks for Link Prediction with Subgraph Sketching](https://mlsanity.com/paper/2209.15486/) (2022)
- [Fine-Grained Spoiler Detection from Large-Scale Review Corpora](https://mlsanity.com/paper/1905.13416/) (2019)
- [Evaluating Graph Neural Networks for Link Prediction: Current Pitfalls and New Benchmarking](https://mlsanity.com/paper/2306.10453/) (2023)
- [A Large-Scale Database for Graph Representation Learning](https://mlsanity.com/paper/2011.07682/) (2020)
- [Multimodal learning with graphs](https://mlsanity.com/paper/2209.03299/) (2022)
- [Knowledge Graphs Meet Multi-Modal Learning: A Comprehensive Survey](https://mlsanity.com/paper/2402.05391/) (2024)
- [GSLB: The Graph Structure Learning Benchmark](https://mlsanity.com/paper/2310.05174/) (2023)
- [GITA: Graph to Visual and Textual Integration for Vision-Language Graph Reasoning](https://mlsanity.com/paper/2402.02130/) (2024)
- [Multimodal Graph Learning for Generative Tasks](https://mlsanity.com/paper/2310.07478/) (2023)
- [Detecting Beneficial Feature Interactions for Recommender Systems](https://mlsanity.com/paper/2008.00404/) (2020)

## Built on by (most cited first, 29 of 29)

- [OpenMAG: A Comprehensive Benchmark for Multimodal-Attributed Graph](https://mlsanity.com/paper/2602.05576/) (2026)
- [MM-OpenFGL: A Comprehensive Benchmark for Multimodal Federated Graph Learning](https://mlsanity.com/paper/2601.22416/) (2026)
- [Toward Effective Multimodal Graph Foundation Model: A Divide-and-Conquer Based Approach](https://mlsanity.com/paper/2602.04116/) (2026)
- [AgentDR: Dynamic Recommendation with Implicit Item-Item Relations via LLM-based Agents](https://mlsanity.com/paper/2510.05598/) (2025)
- [Mario: Multimodal Graph Reasoning with Large Language Models](https://mlsanity.com/paper/2603.05181/) (2026)
- [RoleMAG: Learning Neighbor Roles in Multimodal Graphs](https://mlsanity.com/paper/2604.12271/) (2026)
- [Decoupling and Damping: Structurally-Regularized Gradient Matching for Multimodal Graph Condensation](https://mlsanity.com/paper/2511.20222/) (2025)
- [Multimodal Graph Representation Learning with Dynamic Information Pathways](https://mlsanity.com/paper/2603.09258/) (2026)
- [CAMPA: Efficient and Aligned Multimodal Graph Learning via Decoupled Propagation and Aggregation](https://mlsanity.com/paper/2605.11468/) (2026)
- [LION: A Clifford Neural Paradigm for Multimodal-Attributed Graph Learning](https://mlsanity.com/paper/2608.24795/) (2026)
- [MULTIBENCH++: A Unified and Comprehensive Multimodal Fusion Benchmarking Across Specialized Domains](https://mlsanity.com/paper/2511.06452/) (2025)
- [Modality as Heterogeneity: Node Splitting and Graph Rewiring for Multimodal Graph Learning](https://mlsanity.com/paper/2602.00067/) (2026)
- [GraphVLM: Benchmarking Vision Language Models for Multimodal Graph Learning](https://mlsanity.com/paper/2603.13370/) (2026)
- [Context-aware Modality-Topology Co-Alignment for Multimodal Attributed Graphs](https://mlsanity.com/paper/2606.14172/) (2026)
- [One Model, Many Graphs: Learning over Attributed Graphs across Heterogeneous Modalities with Vision-Language Models](https://mlsanity.com/paper/2607.19128/) (2026)
- [Towards Multimodal Graph Large Language Model](https://mlsanity.com/paper/2506.09738/) (2025)
- [LION: A Clifford Neural Paradigm for Multimodal-Attributed Graph Learning](https://mlsanity.com/paper/2601.21453/) (2026)
- [OptiMAG: Structure-Semantic Alignment via Unbalanced Optimal Transport](https://mlsanity.com/paper/2601.22856/) (2026)
- [Stable Multimodal Graph Unlearning via Feature-Dimension Aware Quantile Selection](https://mlsanity.com/paper/2605.03303/) (2026)
- [GOMA: Toward Structure-Driven Multimodal Alignment from a Graph Signal Smoothing Perspective](https://mlsanity.com/paper/2605.15723/) (2026)
- [Multimodal Graph Negative Learning](https://mlsanity.com/paper/2606.12863/) (2026)
- [SMGFM: Spectral Multimodal Graph Pretraining for Multimodal-Attributed Graphs](https://mlsanity.com/paper/2606.12867/) (2026)
- [Towards Modality-imbalanced Federated Graph Learning: A Data Synthesis-based Approach](https://mlsanity.com/paper/2606.20382/) (2026)
- [TaLK: Text-attributed Graph Dataset Distillation via Coupling Language Model with Graph-Aware Kernel](https://mlsanity.com/paper/2606.22975/) (2026)
- [FedOGL: Combating Catastrophic Forgetting in Federated Open-World Multimodal Graph Learning](https://mlsanity.com/paper/2607.27665/) (2026)
- [ReCoG: Reciprocal Co-Evolution for Multimodal Graph Learning](https://mlsanity.com/paper/2608.22786/) (2026)
- [ICE: Task-Aligned Clifford Latent Fields for Multimodal Graph Foundation Models](https://mlsanity.com/paper/2609.29398/) (2026)
- [GraphSelect for Budgeted Representation Selection in Multimodal Graph Inference](https://mlsanity.com/paper/2609.33561/) (2026)
- [Toward Omni Multimodal Graph Foundation Model: A Topology-Driven Binding Approach](https://mlsanity.com/paper/2610.02881/) (2026)

## Abstract (arXiv)

Graph machine learning has made significant strides in recent years, yet the integration of visual information with graph structure and its potential for improving performance in downstream tasks remains an underexplored area. To address this critical gap, we introduce the Multimodal Graph Benchmark (MM-GRAPH), a pioneering benchmark that incorporates both visual and textual information into graph learning tasks. MM-GRAPH extends beyond existing text-attributed graph benchmarks, offering a more comprehensive evaluation framework for multimodal graph learning Our benchmark comprises seven diverse datasets of varying scales (ranging from thousands to millions of edges), designed to assess algorithms across different tasks in real-world scenarios. These datasets feature rich multimodal node attributes, including visual data, which enables a more holistic evaluation of various graph learning frameworks in complex, multimodal environments. To support advancements in this emerging field, we provide an extensive empirical study on various graph learning frameworks when presented with features from multiple modalities, particularly emphasizing the impact of visual information. This study offers valuable insights into the challenges and opportunities of integrating visual data into graph learning.