mlsanityData to 7 Oct 2026

Mosaic of Modalities: A Comprehensive Benchmark for Multimodal Graph Learning

arXiv 2406.16321 ↗First posted 24 Jun 2024cs.LG cs.AIcited by 28

Not on the map. Why?

Abstract (arXiv)

Graph machine learning has made significant strides in recent years, yet the integration of visual information with graph structure and its potential for improving performance in downstream tasks remains an underexplored area. To address this critical gap, we introduce the Multimodal Graph Benchmark (MM-GRAPH), a pioneering benchmark that incorporates both visual and textual information into graph learning tasks. MM-GRAPH extends beyond existing text-attributed graph benchmarks, offering a more comprehensive evaluation framework for multimodal graph learning Our benchmark comprises seven diverse datasets of varying scales (ranging from thousands to millions of edges), designed to assess algorithms across different tasks in real-world scenarios. These datasets feature rich multimodal node attributes, including visual data, which enables a more holistic evaluation of various graph learning frameworks in complex, multimodal environments. To support advancements in this emerging field, we provide an extensive empirical study on various graph learning frameworks when presented with features from multiple modalities, particularly emphasizing the impact of visual information. This study offers valuable insights into the challenges and opportunities of integrating visual data into graph learning.

Key facts. Cited by 28 papers in the map (data through 7 Oct 2026). Builds on 34 of its 53 references that are in the map. First posted on arXiv 24 Jun 2024.

Builds on

34 of 53 references.

References · 34

  1. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,456
  2. 2020An Image is Worth 16x16 Words: Transformers for Image Recognition at Scalecited by 29,915
  3. 2019Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformercited by 14,491
  4. 2016Semi-Supervised Classification with Graph Convolutional Networkscited by 11,282
  5. 2023DINOv2: Learning Robust Visual Features without Supervisioncited by 7,764
  6. 2017Inductive Representation Learning on Large Graphscited by 6,284
  7. 2019Optuna: A Next-generation Hyperparameter Optimization Frameworkcited by 2,455
  8. 2020Open Graph Benchmark: Datasets for Machine Learning on Graphscited by 2,148
  9. 2023ImageBind: One Embedding Space To Bind Them Allcited by 1,162
  10. 2023Large Language Modelscited by 774
  11. 2024Eyes Wide Shut? Exploring the Visual Shortcomings of Multimodal LLMscited by 719
  12. 2020TUDataset: A collection of benchmark datasets for learning with graphscited by 718
22 more
  1. 2021FLAVA: A Foundational Language And Vision Alignment Modelcited by 646
  2. 2022Multimodal Learning with Transformers: A Surveycited by 399
  3. 2022Long Range Graph Benchmarkcited by 258
  4. 2024Bridging Language and Items for Retrieval and Recommendation: Benchmarking LLMs as Semantic Encoderscited by 213
  5. 2023Large Language Models on Graphs: A Comprehensive Surveycited by 190
  6. 2022GOOD: A Graph Out-of-Distribution Benchmarkcited by 140
  7. 2019MMKG: Multi-Modal Knowledge Graphscited by 104
  8. 2020CoDEx: A Comprehensive Knowledge Graph Completion Benchmarkcited by 102
  9. 2022Graph Neural Networks for Link Prediction with Subgraph Sketchingcited by 81
  10. 2019Fine-Grained Spoiler Detection from Large-Scale Review Corporacited by 76
  11. 2023Evaluating Graph Neural Networks for Link Prediction: Current Pitfalls and New Benchmarkingcited by 69
  12. 2020A Large-Scale Database for Graph Representation Learningcited by 54
  13. 2022Multimodal learning with graphscited by 54
  14. 2024Knowledge Graphs Meet Multi-Modal Learning: A Comprehensive Surveycited by 50
  15. 2023GSLB: The Graph Structure Learning Benchmarkcited by 49
  16. 2024GITA: Graph to Visual and Textual Integration for Vision-Language Graph Reasoningcited by 31
  17. 2023Multimodal Graph Learning for Generative Taskscited by 27
  18. 2020Detecting Beneficial Feature Interactions for Recommender Systemscited by 22
  19. 2022MoSE: Modality Split and Ensemble for Multimodal Knowledge Graph Completioncited by 14
  20. 2023Pitfalls in Link Prediction with Graph Neural Networks: Understanding the Impact of Target-link Inclusion & Better Practicescited by 13
  21. 2023TouchUp-G: Improving Feature Representation through Graph-Centric Finetuningcited by 12
  22. 2022Are Message Passing Neural Networks Really Helpful for Knowledge Graph Completion?cited by 6

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2025: 4; 2026: 24.

28 papers.

Sort:

By area:

  1. 2026MM-OpenFGL: A Comprehensive Benchmark for Multimodal Federated Graph Learningcited by 11
  2. 2026OpenMAG: A Comprehensive Benchmark for Multimodal-Attributed Graphcited by 11
  3. 2026Toward Effective Multimodal Graph Foundation Model: A Divide-and-Conquer Based Approachcited by 6
  4. 2025AgentDR: Dynamic Recommendation with Implicit Item-Item Relations via LLM-based Agentscited by 3
  5. 2025Decoupling and Damping: Structurally-Regularized Gradient Matching for Multimodal Graph Condensationcited by 2
  6. 2026Mario: Multimodal Graph Reasoning with Large Language Modelscited by 2
  7. 2026Multimodal Graph Representation Learning with Dynamic Information Pathwayscited by 2
  8. 2026RoleMAG: Learning Neighbor Roles in Multimodal Graphscited by 2
  9. 2026CAMPA: Efficient and Aligned Multimodal Graph Learning via Decoupled Propagation and Aggregationcited by 2
  10. 2026LION: A Clifford Neural Paradigm for Multimodal-Attributed Graph Learningcited by 2
  11. 2025MULTIBENCH++: A Unified and Comprehensive Multimodal Fusion Benchmarking Across Specialized Domainscited by 1
  12. 2026Modality as Heterogeneity: Node Splitting and Graph Rewiring for Multimodal Graph Learningcited by 1
16 more
  1. 2026GraphVLM: Benchmarking Vision Language Models for Multimodal Graph Learningcited by 1
  2. 2026Context-aware Modality-Topology Co-Alignment for Multimodal Attributed Graphscited by 1
  3. 2026One Model, Many Graphs: Learning over Attributed Graphs across Heterogeneous Modalities with Vision-Language Modelscited by 1
  4. 2025Towards Multimodal Graph Large Language Modelcited by 0
  5. 2026LION: A Clifford Neural Paradigm for Multimodal-Attributed Graph Learningcited by 0
  6. 2026OptiMAG: Structure-Semantic Alignment via Unbalanced Optimal Transportcited by 0
  7. 2026Stable Multimodal Graph Unlearning via Feature-Dimension Aware Quantile Selectioncited by 0
  8. 2026GOMA: Toward Structure-Driven Multimodal Alignment from a Graph Signal Smoothing Perspectivecited by 0
  9. 2026Multimodal Graph Negative Learningcited by 0
  10. 2026SMGFM: Spectral Multimodal Graph Pretraining for Multimodal-Attributed Graphscited by 0
  11. 2026Towards Modality-imbalanced Federated Graph Learning: A Data Synthesis-based Approachcited by 0
  12. 2026TaLK: Text-attributed Graph Dataset Distillation via Coupling Language Model with Graph-Aware Kernelcited by 0
  13. 2026FedOGL: Combating Catastrophic Forgetting in Federated Open-World Multimodal Graph Learningcited by 0
  14. 2026ReCoG: Reciprocal Co-Evolution for Multimodal Graph Learningcited by 0
  15. 2026ICE: Task-Aligned Clifford Latent Fields for Multimodal Graph Foundation Modelscited by 0
  16. 2026GraphSelect for Budgeted Representation Selection in Multimodal Graph Inferencecited by 0