mlsanityData to 7 Oct 2026

Modality as Heterogeneity: Node Splitting and Graph Rewiring for Multimodal Graph Learning

arXiv 2602.00067 ↗First posted 20 Jan 2026cs.LG cs.AIcited by 1

On the map: Mixture-of-experts → Multimodal MoE models

Abstract (arXiv)

Multimodal graphs are gaining increasing attention due to their rich representational power and wide applicability, yet they introduce substantial challenges arising from severe modality confusion. To address this issue, we propose NSG (Node Splitting Graph)-MoE, a multimodal graph learning framework that integrates a node-splitting and graph-rewiring mechanism with a structured Mixture-of-Experts (MoE) architecture. It explicitly decomposes each node into modality-specific components and assigns relation-aware experts to process heterogeneous message flows, thereby preserving structural information and multimodal semantics while mitigating the undesirable mixing effects commonly observed in general-purpose GNNs. Extensive experiments on three multimodal benchmarks demonstrate that NSG-MoE consistently surpasses strong baselines. Despite incorporating MoE -- which is typically computationally heavy -- our method achieves competitive training efficiency. Beyond empirical results, we provide a spectral analysis revealing that NSG performs adaptive filtering over modality-specific subspaces, thus explaining its disentangling behavior. Furthermore, an information-theoretic analysis shows that the architectural constraints imposed by NSG reduces mutual information between data and parameters and improving generalization capability.

Key facts. Cited by 1 paper in the map (data through 7 Oct 2026). Placed in Multimodal MoE models, one of 12 directions in Mixture-of-experts. Builds on 47 of its 55 references that are in the map. First posted on arXiv 20 Jan 2026.

Builds on

47 of 55 references.

Closest ancestors · 1

Same direction, newest first

  1. 2025UniGraph2: Learning a Unified Embedding Space to Bind Multimodal Graphs2 Feb

Other references · 46

  1. 2015Deep Residual Learning for Image Recognitioncited by 51,809
  2. 2014Adam: A Method for Stochastic Optimizationcited by 36,960
  3. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,456
  4. 2020An Image is Worth 16x16 Words: Transformers for Image Recognition at Scalecited by 29,915
  5. 2016Semi-Supervised Classification with Graph Convolutional Networkscited by 11,282
  6. 2019Sentence-BERT: Sentence Embeddings using Siamese BERT-Networkscited by 9,049
  7. 2017Graph Attention Networkscited by 8,854
  8. 2017Inductive Representation Learning on Large Graphscited by 6,284
  9. 2017Outrageously Large Neural Networks: The Sparsely-Gated Mixture-of-Experts Layercited by 3,413
  10. 2019A Comprehensive Survey on Graph Neural Networkscited by 2,906
  11. 2019ViLBERT: Pretraining Task-Agnostic Visiolinguistic Representations for Vision-and-Language Taskscited by 2,219
  12. 2017Modeling Relational Data with Graph Convolutional Networkscited by 1,684
34 more
  1. 2019LXMERT: Learning Cross-Modality Encoder Representations from Transformerscited by 1,548
  2. 2019Simplifying Graph Convolutional Networkscited by 1,470
  3. 2016Variational Graph Auto-Encoderscited by 1,373
  4. 2023ImageBind: One Embedding Space To Bind Them Allcited by 1,162
  5. 2018Graph Convolutional Neural Networks for Web-Scale Recommender Systemscited by 1,133
  6. 2018Representation Learning on Graphs with Jumping Knowledge Networkscited by 875
  7. 2020Simple and Deep Graph Convolutional Networkscited by 857
  8. 2019Heterogeneous Graph Attention Networkcited by 732
  9. 2019DropEdge: Towards Deep Graph Convolutional Networks on Node Classificationcited by 679
  10. 2020Heterogeneous Graph Transformercited by 581
  11. 2022GraphMAE: Self-Supervised Masked Graph Autoencoderscited by 443
  12. 2017Information-theoretic analysis of generalization capability of learning algorithmscited by 299
  13. 2018Modeling polypharmacy side effects with graph convolutional networkscited by 237
  14. 2022NAGphormer: A Tokenized Graph Transformer for Node Classification in Large Graphscited by 137
  15. 2020GraphNorm: A Principled Approach to Accelerating Graph Neural Network Trainingcited by 116
  16. 2021Mining Latent Structures for Multimedia Recommendationcited by 79
  17. 2023Graph Mixture of Experts: Learning on Large-Scale Graphs with Explicit Diversity Modelingcited by 63
  18. 2022Multimodal learning with graphscited by 54
  19. 2020Interpretable and Efficient Heterogeneous Graph Convolutional Networkcited by 48
  20. 2022Hybrid Transformer with Multi-level Fusion for Multimodal Knowledge Graph Completioncited by 44
  21. 2023LGMRec: Local and Global Graph Learning for Multimodal Recommendationcited by 37
  22. 2022Multi-modal Graph Learning for Disease Predictioncited by 34
  23. 2024Mosaic of Modalities: A Comprehensive Benchmark for Multimodal Graph Learningcited by 28
  24. 2022Multiplex Heterogeneous Graph Convolutional Networkcited by 27
  25. 2023Multimodal Graph Learning for Generative Taskscited by 27
  26. 2024GraphMoRE: Mitigating Topological Heterogeneity via Mixture of Riemannian Expertscited by 23
  27. 2024When Graph meets Multimodal: Benchmarking and Meditating on Multimodal Attributed Graphs Learningcited by 20
  28. 2024Graph Mixture of Experts and Memory-augmented Routers for Multivariate Time Series Anomaly Detectioncited by 19
  29. 2023A Suite of Generative Tasks for Multi-Level Multimodal Webpage Understandingcited by 18
  30. 2021Graph Classification by Mixture of Diverse Expertscited by 16
  31. 2022Heterogeneous Graph Learning for Multi-modal Medical Data Analysiscited by 16
  32. 2022Convolutional Learning on Multigraphscited by 10
  33. 2023Spectral Heterogeneous Graph Convolutions via Positive Noncommutative Polynomialscited by 8
  34. 2025Multimodal Graph-Based Variational Mixture of Experts Network for Zero-Shot Multimodal Information Extractioncited by 3

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2026: 1.

1 paper.

  1. 2026ReCoG: Reciprocal Co-Evolution for Multimodal Graph Learningcited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2026Towards Robust Federated Multimodal Graph Learning under Modality Heterogeneity
  2. 2025Graph4MM: Weaving Multimodal Learning with Structural Information
  3. 2026GraphVLM: Benchmarking Vision Language Models for Multimodal Graph Learning
  4. 2025REMOTE: A Unified Multimodal Relation Extraction Framework with Multilevel Optimal Transport and Mixture-of-Experts
  5. 2025Graph-MLLM: Harnessing Multimodal Large Language Models for Multimodal Graph Learning
  6. 2026MGDT: MLLM-Guided Diffusion Transformer with Relation-Adaptive Mixture-of-Experts for Multimodal Knowledge Graph Completion
  7. 2025DiffusionCom: Structure-Aware Multimodal Diffusion Model for Multimodal Knowledge Graph Completion
  8. 2026MURAL: Multimodal Uncertainty-aware Recommendation via Adaptive edge Learning